Parts manipulation, inspection, and replacement

ABSTRACT

Manufacturing lines include inspection systems for monitoring the quality of parts produced. Manufacturing lines for making semiconductor devices generally inspect each fabricated part. The information obtained is used to fix manufacturing problems in the semiconductor fab plant. A machine-vision system for inspecting devices includes a flipper mechanism. After being inspected at a first station, a tray-transfer device moves the tray from the first inspection station to a flipper mechanism. The flipper mechanism includes two jaws, a mover, and a rotator. The flipper mechanism turns the devices over and places the devices in a second tray so that another surface of the device can be inspected. A second tray-transfer device moves the second tray from the flipper to a second inspection station. The mover of the flipper mechanism removes the tray from the first inspection surface and places a tray at the second inspection surface.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a divisional of, and claims priority to, U.S. patent applicationSer. No. 13/297,227 filed Nov. 15, 2011, titled “PARTS MANIPULATION,INSPECTION, AND REPLACEMENT SYSTEM AND METHOD” (to issue as U.S. Pat.No. 8,286,780 on Oct. 16, 2012); which claimed priority to U.S. patentapplication Ser. No. 12/099,774 filed Apr. 8, 2008, titled “TRAY FLIPPERAND METHOD FOR PARTS INSPECTION” (issued as U.S. Pat. No. 8,056,700 onNov. 15, 2011); which claimed priority to U.S. patent application Ser.No. 09/350,251 filed Jul. 8, 1999, titled “TRAY FLIPPER AND METHOD FORPARTS INSPECTION” (issued as U.S. Pat. No. 7,353,954 on Apr. 8, 2008);which claimed priority to U.S. Provisional Patent Application No.60/092,089 filed Jul. 8, 1998; each of which is incorporated herein byreference in its entirety.

The present invention is related to:

U.S. patent application Ser. No. 09/757,834, filed Jul. 8, 1999, titled“IMAGING FOR A MACHINE VISION SYSTEM” (now U.S. Pat. No. 6,956,963);U.S. patent application Ser. No. 09/350,051, titled “CIRCUIT FORMACHINE-VISION SYSTEM” (now U.S. Pat. No. 6,603,103);U.S. patent application Ser. No. 09/350,050, titled “MACHINE-VISIONSYSTEM AND METHOD FOR RANDOMLY LOCATED PARTS” (now abandoned);U.S. patent application Ser. No. 09/350,255, titled “PARTS MANIPULATIONAND INSPECTION SYSTEM AND METHOD” (now abandoned);U.S. patent application Ser. No. 09/349,684, titled “MACHINE-VISIONSYSTEMS AND METHODS WITH UP AND DOWN LIGHTS” (now abandoned);U.S. patent application Ser. No. 09/757,752, filed Jul. 8, 1999, titled“IDENTIFYING AND HANDLING DEVICE TILT IN A THREE-DIMENSIONALMACHINE-VISION IMAGE,”U.S. patent application Ser. No. 09/349,948, entitled “Identifying andHandling Device Tilt in a Three-Dimensional Machine Vision Image” (nowabandoned);U.S. patent application Ser. No. 09/350,049, titled “COMBINED 3D-AND2D-SCANNING MACHINE-VISION SYSTEM AND METHOD” (now U.S. Pat. No.6,522,777); andU.S. patent application Ser. No. 09/350,037, titled “Parts Manipulationand Inspection System and Method” (now abandoned);each of which is incorporated herein by reference in its entirety.

FIELD OF THE INVENTION

This invention relates to the field of machine vision, and morespecifically to a mechanical apparatus and methods used to obtainthree-dimensional inspection. The present invention relates to a deviceto flip trays or other containers of devices by a machine. The trayflipping device may be part of a machine-vision-inspection station on anautomated manufacturing line.

BACKGROUND OF THE INVENTION

There is a widespread need for inspection data for electronic parts in amanufacturing environment. One common inspection method uses a videocamera to acquire two-dimensional images of a device-under-test.

Height distribution of a surface can be obtained by projecting a lightstripe pattern onto the surface and then reimaging the light patternthat appears on the surface. One technique for extracting thisinformation based on taking multiple images (3 or more) of the lightpattern that appears on the surface while shifting the position (phase)of the projected light stripe pattern is referred to as phase-shiftinginterferometry, as disclosed in U.S. Pat. Nos. 4,641,972 and 4,212,073(incorporated herein by reference).

The multiple images are usually taken using a CCD (charge-coupleddevice) video camera with the images being digitized and transferred toa computer where phase-shift analysis, based on images being used as“buckets,” converts the information to a contour map (i.e., athree-dimensional representation) of the surface.

The techniques used to obtain the multiple images are based on methodsthat keep the camera and viewed surface stationary with respect to eachother while moving the projected pattern.

One technique for capturing just one bucket image using a line scancamera is described in U.S. Pat. No. 4,965,665 (incorporated herein byreference).

U.S. Pat. Nos. 5,398,113 and 5,355,221 (incorporated herein byreference) disclose white-light interferometry systems which profilesurfaces of objects.

In U.S. Pat. No. 5,636,025 (incorporated herein by reference), anoptical measuring system is disclosed which includes a light source,gratings, lenses, and camera. A mechanical translation device moves oneof the gratings in a plane parallel to a reference surface to effect aphase shift of a projected image of the grating on the contoured surfaceto be measured. A second mechanical translation device moves one of thelenses to effect a change in the contour interval. A first phase of thepoints on the contoured surface is taken, via a four-bucket algorithm,at a first contour interval. A second phase of the points is taken at asecond contour interval. A control system, including a computer,determines a coarse measurement using the difference between the firstand second phases. The control system further determines a finemeasurement using either the first or second phase. The displacement ordistance, relative to the reference plane, of each point is determined,via the control system, using the fine and coarse measurements.

Current vision-inspection systems have many problems. Among the problemsare assorted problems associated with current mechanical translationdevices associated with vision-inspection systems. Among the mechanicaltranslation problems are that the trays of devices, such as trays ofsemiconductor chips, associated with current vision-inspection systemsmust be flipped by hand since both sides of the devices must beinspected. In current systems, the flipping of trays of devices is doneby an operator. Operator intervention requires time and may be prone toerror. Throughput of the machine-vision-inspection system may also beaffected. For example, if the operator is on break or otherwiseunavailable, throughput of the machine-vision-inspection system sufferssince the machine is stopped awaiting operator intervention. Asmentioned, errors can also occur as a direct result of operatorintervention. When both sides of the part need to be inspected the partsmust be flipped. If the operator is interrupted while flipping thetrays, the operator may forget the orientation of the devices or traysafter the interruption. In other words, due to the interruption, thetray carrying the devices may not be flipped. At best, the subsequentoperation will detect that the devices are not flipped if themachine-vision-inspection system has the capability to detect that thewrong portion of the device is presented for inspection. In this event,more time is lost since the operator must now flip the unflipped trays.This of course also negatively affects throughput of themachine-vision-inspection system. At worst, themachine-vision-inspection system would not detect that the wrong side ofthe device (same side of the device previously inspected) was againbeing inspected. In other words, the inspection would be repeated. Theresult would be far worse in that devices that would not pass theinspection would be shipped to customers or placed in larger devicesshipped to customers. If the products are caught by a subsequentsampling, the larger devices would have to be reworked. If the productswere not caught be a subsequent sampling or inspection, defectiveproduct may be shipped to a customer. This of course is devastating in aworld where product quality is constantly stressed by marketingexecutives. Even a hint of less than top quality can devastate themarket for a product.

To overcome the problems stated above as well as other problems, thereis a need for an improved machine-vision system and more specificallyfor a machine-vision-inspection system capable of flipping trays ofdevices without operator intervention. In addition, there is a need fora flipping device that operates to reliably flip the trays of devices sothat all portions of the devices within the trays are reliablyinspected. In addition, there is a need for a flipping device thatfacilitates automated high-speed three-dimensional inspection of objectsin a manufacturing environment. There is also a need for a device thatis easily accommodated as a station on an automated manufacturing line.

BRIEF SUMMARY OF THE INVENTION

A machine-vision system for inspecting a device monitors the quality ofmanufactured parts or other devices produced. A machine-vision systemfor inspecting a device with a first side and a second side includes afirst inspection station for inspecting a first side of a device and asecond inspection station for inspecting a second side of a device. Themachine-vision system also includes a tray-transfer device that operatesto move the device from the first inspection station to the secondinspection station. The tray-transfer device also includes an invertingmechanism or flipper mechanism that operates to invert the device sothat the first second side of the device can be inspected at the firstinspection station and the second side of the device can be inspected atthe second inspection station. After being inspected at a first station,a conveyor moves the tray from the first inspection station to a flippermechanism. The flipper mechanism includes two jaws and a rotator. Thefirst tray is loaded onto a surface of one of the jaws. A second tray isheld by a second jaw. Once the first tray is loaded, the first andsecond jaw and the first tray and second tray are moved into engagementwith one another. The flipper mechanism then rotates which turns thedevices over and places the devices in a second tray so that anothersurface of the device can be inspected. A second conveyor moves thesecond tray from the flipper to a second inspection station.

The trays used to hold devices are rectangular in shape and are placedon a conveyor so that the trays travel in a direction parallel to theshortest dimension of the tray. In other words, the length of theconveyor is shortened since more trays can be fit on a shorter conveyormechanism. The footprint of the machine-vision system is smaller thancurrent machine-vision systems.

A method for acquiring physical information associated with of aplurality of includes the steps of loading at least one tray into acompartment near the first inspection station. The tray is elevated tothe first inspection surface associated with a first inspection stationwhere the first side of a device within a first tray is inspected. Onceinspected, the tray is moved to the flip station. At the flip station,the devices within the first tray are flipped and placed within a secondtray. A conveyor moves the second tray from the flip station to a secondinspection station where the second side of the device within the secondtray is inspected. If a device within the second tray fails theinspection of either the first side or the second side of the device,the failed device is removed from the second tray and replaced with adevice that passed inspection.

Advantageously, the machine-vision-inspection system is capable offlipping trays of devices without operator intervention. The flippingdevice operates to reliably flip the trays of devices so that allportions of the devices within the trays are reliably inspected. Theflipping device that facilitates automated high-speed three-dimensionalinspection of objects in a manufacturing environment. The flippingmechanism is also easily accommodated as a station on an automatedmanufacturing line. Yet another advantage is that the inspection cantake place automatically without the intervention of a human. Thislessens the chance for operator error during the inspection process andaids in the throughput of the machine-vision-inspection system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an embodiment of the present invention, a system 100 forthe manufacture and inspection of devices.

FIG. 2 shows an embodiment of the present invention, a computercontrolled system 200 for the control of the imaging operation andmeasurement functions of system 100.

FIG. 3 shows an overview of scanning head 401.

FIG. 4A shows one embodiment of a machine-vision head 401 for inspectinga device 99.

FIG. 4B shows another embodiment of a machine-vision head 401.

FIG. 4C shows yet another embodiment of a machine-vision head 401.

FIG. 4D shows still another embodiment of a machine-vision head 401.

FIG. 4E shows a projection pattern element 412 having a density pattern472 that is a sine-wave in one direction.

FIG. 4F shows a projection pattern element 412′ of another embodiment,and represents a square-wave pattern near the element.

FIG. 4G shows a projection pattern element 412″ of another embodiment,and represents a square-wave pattern near the element.

FIG. 4H shows a projection pattern element 412″ of another embodiment,and represents a square-wave pattern near the element.

FIG. 4I shows yet another embodiment of a machine-vision head 401.

FIG. 5A shows a solder ball 97, illustrating the various illuminated andimagable regions.

FIG. 5B is a representation of light gathered by a non-telecentric lens420.

FIG. 5C is a representation of light gathered by a telecentric lens 420.

FIG. 6 shows machine-vision system 600 that represents anotherembodiment of the present invention having more than one projector in ascanning head.

FIG. 7 shows machine-vision system 700 that represents anotherembodiment of the present invention having more than one imager in ascanning head.

FIG. 8 shows machine-vision system 800 that represents anotherembodiment of the present invention having more than one projector andmore than one imager in a scanning head.

FIG. 9A shows a sensor 904A having a beamsplitter 820A.

FIG. 9B shows a sensor 904B having a beamsplitter 820B.

FIG. 9C shows a sensor 904C having a beamsplitter 820C.

FIG. 10 shows a modular machine-vision system 1000 of one embodiment ofthe present invention.

FIG. 11 shows a computation and comparison system 1100 of one embodimentof the present invention.

FIG. 12 is a schematic layout of another preferred embodiment of thevision system.

FIG. 13A is a schematic view of one preferred embodiment of a lightintensity controller.

FIG. 13B is a schematic view of another preferred embodiment of a lightintensity controller.

FIG. 13C is a schematic view of one preferred embodiment of a lightintensity controller.

FIG. 14A is a schematic view of the imaging system having a memorydevice associated with the trilinear array.

FIG. 14B is a table lookup which is housed within memory and used toapply correction values to the values associated with the pixels of atrilinear array.

FIG. 14C is a schematic view of the imaging system having a memorydevice associated with the trilinear array in which a value associatedwith the intensity of the light is used to correct the values in memory.

FIG. 15 is a schematic view of the trilinear array with a thermoelectriccooling element associated therewith.

FIG. 16 is a perspective view of the machine-vision system.

FIG. 17A is a top view of the compartment and the elevator mechanismsbelow each of the inspection stations of the machine-vision system.

FIG. 17B is a side view of one of the compartment and the elevatormechanisms below an inspection station.

FIG. 18A is a top view of the inspection stations and the tray-transferdevices for moving trays between the various inspection stations, thepick and place stations, and the tray inverter or flipper mechanism.

FIG. 18B is a front view of one of the tray-transfer devices for movingtrays between the various inspection stations, the pick and placestations and the tray inverter or flipper mechanism.

FIG. 19A is a side view of one tray inverter mechanism.

FIG. 19B is a front view of the tray inverter mechanism of FIG. 19A.

FIGS. 19C, 19D, 19E, 19F, 19G are front views of another tray invertermechanism.

FIGS. 19H, 19I, 19J, 19K, 19L are side views of the tray invertermechanism of FIGS. 19C-19G, respectively.

FIG. 20 shows picker for replacing devices.

FIG. 21 shows an acquired image showing various heights of a ball beinginspected.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of the preferred embodiments,reference is made to the accompanying drawings which form a part hereof,and in which are shown by way of illustration specific embodiments inwhich the invention may be practiced. It is to be understood that otherembodiments may be utilized and structural changes may be made withoutdeparting from the scope of the present invention.

Machine-vision and optical-feature-recognition techniques can be used todistinguish parts that deviate from a predetermined intended aspect ofan ideal device. In this description, a “device” is meant to be anydevice of manufacture or object, for example an integrated circuitpackage, electronic part, semiconductor, molded plastic part, aluminumwheel, gemstone or even an egg or strawberry, which can be inspected.Typically, according to the present invention, a manufacturing operationwill use two-dimensional and three dimensional information acquired frominspection of the device to distinguish “good” parts from “bad” parts,and can discard the bad parts and insert previously inspected good partsin their place. The devices under test are placed into pocketed trays orinto cartons for ease of handling and transport, and inspection willtake place of the devices while the devices are in the pocketed trays,according to the present invention.

U.S. Pat. No. 5,646,733 to Bieman (incorporated herein by reference)describes a method and system that include an optical head which ismoved relative to an object at a machine-vision station. A projectedpattern of light (e.g., a pattern of stripes or lines) is scanned acrossthe surface of an object to be inspected to generate an imagable lightsignal to acquire three-dimensional information associated with theobject. The optical head includes at least one pattern projector whichprojects a pattern of lines and an imaging subsystem which includes atrilinear-array camera as a detector. The camera and the at least onepattern projector are maintained in fixed relation to each other. Thetrilinear-array camera includes three linear detector elements, eachhaving for example about 1000 to 4000 pixels, which extend in adirection parallel with the pattern of lines. The geometry of theoptical head is arranged in such a way that each linear detector elementpicks up a different phase in the line pattern. As the optical head isscanned across the surface of interest, the detector elements arecontinuously read out. Depth at each point on the surface is calculatedfrom the intensity readings obtained from each of the detector elementsthat correspond to the same point on the surface. In this way, thephases of the pattern are calculated from the three intensity readingsobtained for each point.

System Overview

FIG. 1 is a schematic representation of a high-speed automatedinspection system 100 and its associated method, according to oneembodiment of the present invention. At station 110, a manufacturingprocess or step produces or modifies a device. In one embodiment, thedevice 99, along with a plurality of other devices are placed in apocketed tray. In other embodiments, other containers are used. In stillother embodiments, devices to be inspected are attached to continuousplastic strips that are, e.g., unwound from a reel in order to beinspected, inspected in line as the devices move across the camera, andthen rewound onto a reel after inspection. In yet other embodiments,devices are moved on a conveyor-belt line under the scanning camera,with no tray, strip, or other container. Thus, wherever the inspectionof devices in trays is discussed herein, it is to be understood thatother embodiments that inspect devices using other containers, ordevices attached to strips, or even devices without any container, arecontemplated as alternative embodiments.

At station 115, the tray of devices is scanned to acquiretwo-dimensional (2D) and/or three-dimensional (3D) data representing thegeometry of every device in the tray. In one embodiment, a CCD camerahaving digital output is used in station 115. In one such embodiment,the digital output of the CCD represents a 2D image of device 99. Inanother embodiment, a 3D scanning Moiré interferometry sensor such assensor 400A of FIG. 4A is used to acquire 3D dimensions (i.e., the X, Y,and Z dimensions of various features of device 99), and/or intensitymeasurements (i.e., the brightness, color, or reflectivity of variousfeatures of device 99). The acquired 2D and/or 3D and/or intensity datais processed at station 120, and compared to data of a predeterminedintended geometry. In one embodiment, this comparison distinguishes gooddevices from bad devices, and a signal 121 is output. In one embodiment,signal 121 is used at station 125 as a control to compensate for thedetected departure from the intended geometry, thus providing feedbackinto the manufacturing step at station 110, in order to improve qualityand yield of the manufacturing system. In another embodiment, signal 121is used at station 130 to control the replacement of defective orsubstandard parts with good parts from a previously inspected tray ofparts. At station 135, trays or containers of all-good parts are outputfrom the system.

In one such exemplary system, at station 110, semiconductor parts (suchas microprocessors) are manufactured, wherein the semiconductor partshave one major surface (the “connector side”) that has a pattern ofsolder-ball connectors (a ball-grid array, or BGA). It is desirable tohave the balls each located at a predetermined X and Y coordinate, andeach having a predetermined Z-dimension height (within a predeterminedtolerance) such that the “tops” of the balls are coplanar to within apredetermined tolerance. It is also desirable to have the substrate thatthe balls are connected to also be planar. The major surface oppositethe connector side (the “label side”) is typically marked with anidentification label. In one such embodiment, the devices are inspectedat inspection station 115 on their connector side, then the devices areflipped over (e.g., into another tray) and inspected on their labelside. In this embodiment, the inspected tray of parts are then passed tothe replacement station 130, and a pick-and-place mechanism (in oneembodiment, a vacuum-actuated robotic arm) removes defective partsaccording to data from signal 121, and replaces them with good partsfrom a previously inspected, partially filled tray of good parts. Thustrays having complete populations of all-good parts are output atstation 135.

In another such exemplary system, at station 110, objects to beinspected (for example eggs) are placed into pocketed trays (forexample, egg cartons). At station 115, the objects are inspected (e.g.,for size, shape, and visible defects for example blotches or cracks). Insuch a system, the feedback and control through station 125 are omitted.Signal 121 is used to control the replacement of defective objects,and/or the sorting of objects into trays or containers according tosize, shape, color, or other criteria. Thus the present invention can beused both in manufacturing environments as well as in the sorting andpackaging of non-manufactured objects such as eggs or gemstones whichmay be collected from nature or other non-manufactured source.

FIG. 2 shows one exemplary embodiment of the present invention, acomputer controlled system 200 for the control of the imaging operationand measurement functions of system 100. Host computer 128 is coupledthrough system bus 126 to mass storage unit 132 (e.g., a magnetic disksubsystem), input/output (I/O) subsystem 130, imager interface 122, anddisplay 134. In one embodiment, imager interface 122 is coupled to anoptical head 401 such as shown in FIG. 4A. In one embodiment, I/Osubsystem 130 controls drive 131 which moves either optical head 401 ordevice 99 or both, in order to obtain a relative scanning motion betweenthe two (one embodiment which scans trays of devices 99 keeps the traysstationary during the optical scanning operation, and moves only opticalhead 401, in order to eliminate movement of devices 99 within the traydue to vibration from the motors, and thereby obtain more accuratemeasurements). In one such embodiment, drive 131 moves head 401 to firstscan one or more entire rows of devices 99 in a tray in a single pass inthe Y direction, then increments in the X direction, then performs asecond scan of one or more entire rows of devices 99 in a single pass inthe Y direction (parallel to but in the opposite direction to the firstscan), and repeats this until the entire tray of parts is measured.Since each scan obtains data from a relatively wide scan stripe, thereis no need to vibrate the tray to align the parts to one side of thetray pockets, or even necessarily to have tray pockets, although pocketsare provided in one embodiment in order to facilitate other steps, suchas pick-and-place operations to remove bad parts from trays and replacethem with good parts. Imager interface 122 obtains raw scan data fromoptical head 401, and computes and stores processed data representativeof the intensity and/or height of each X and Y coordinate of each device99 (called “grabbing a frame”). Based on a comparison of the measureddata to a predetermined set or range of acceptable or desired data, hostcomputer 128 controls signal 121 through I/O subsystem 130. Signal 121is used (in various embodiments) to control a step in the manufacturingprocess and/or to select a set of “all good” parts from among all partsproduced.

FIG. 3 shows an overview of 3D scanning head 401. Scanning head 401includes pattern projector 402 and imager 404. Pattern projector 402emits a pattern of light 499 that varies in intensity along a line inthe Z dimension (e.g., along the optical axis of imager 404) and along aline in the Y dimension (i.e., along the direction of travel 90). In onesuch embodiment, a striped Moiré-type pattern is projected, wherein thepattern varies as a sine wave in the Y and Z dimensions, but for each Yand Z value, is constant in the X dimension. Imager 404 includes imagingelement 420 that focuses an image of the device 99 onto the three linesof semiconductor imaging pixels of trilinear array 423 of detector 422.Each of the three lines of semiconductor imaging pixels of trilineararray 423 include a plurality of detecting pixels (e.g., 2048 pixelseach) that are adjacent to one another in the X dimension (detector 422is shown in an isometric projection in FIG. 3), and the three lines ofsemiconductor imaging pixels are separated from each other by a distancein the Y dimension of, for example, 8 times the X-dimensionpixel-to-pixel spacing. Each point being measured on device 99 is imagedthree times by moving scanning head 401 relative to device 99 in the +Yor −Y direction (e.g., in direction 90, or in the opposite direction),and capturing a data point from a pixel of the first line ofsemiconductor imaging pixels 423, and then on the corresponding pixel ofthe second and third line of semiconductor imaging pixels 423. Thepattern projector 402 and the imager 404 are maintained in substantiallyfixed relation to each other (i.e., the projection pattern is notshifted or moved relative to the pattern projector 402 or the imager404), and thus high measurement accuracy can be maintained. Thegeometric spacing of the light pattern 499 relative to the spacingsbetween the three lines of semiconductor imaging pixels 423 is designedsuch that three different intensity readings are obtained for eachmeasured point (i.e., the periodicity of light pattern 499 should notcorrespond exactly to spacings between the three lines of semiconductorimaging pixels 423).

In this embodiment, the head is moved in the Y direction (by a distanceequal to the center-to-center X-direction-spacings of pixels betweeneach image line detection operation), perpendicular to the pixel lines423, so the corresponding pixel of each of the three lines ofsemiconductor imaging pixels 423 will measure the same point {x,y,z} inthree time periods. Also in this embodiment, the pixel lines 423 areseparated center-to-center by a distance equal to eight times thecenter-to-center X-direction-spacings of pixels. Thus each point {x,y,z}will be detected by a pixel in the first line of the three lines ofsemiconductor imaging pixels 423, then eight line-measurement clockslater, it will be detected by the corresponding pixel in the second lineof the three lines of semiconductor imaging pixels 423, and then eightline-measurement clocks later, it will be detected by the correspondingpixel in the third line. (At each “line-measurement clock,” each of the,for example, 2048 detector pixels of each line of the three lines ofsemiconductor imaging pixels 423 will acquire the image of one spot ofdevice 99, and an analog signal representing that image is transferredto a charge-coupled device (CCD) shift register, and the three sets of2048 values each are shifted out as three parallel stream of 2048 serialanalog values, and are each converted to one of three sets of 2048digital values. By analyzing the difference between the first and secondreadings (displaced in this example by eight line-measurement clocks)and the difference between the second and third readings (againdisplaced in this example by eight more line-measurement clocks), theheight (Z) of each point or spot is derived.

The function of the layout of FIG. 3 and FIG. 4 (i.e., FIGS. 4A-41) isto project a sine-wave “Moiré” pattern onto an object or device 99 andto measure each point at three places along the scan, for example,obtaining three reflected intensities with a tri-linear CCD (without thetypical red/green/blue color filters).

In one embodiment, imager 404 is a Dalsa camera, and the CCD 423 used bythe Dalsa camera is the Kodak KLI-2103 which contains 3 rows of 2098active photo sites (in another embodiment, the Tri-Linear CCD, partnumber KLI-2113, from Eastman Kodak Co, Rochester, N.Y. is used). Eachphotosite of the Kodak KLI-2103 measures 14 μm square and the center tocenter spacing between the rows is 112 μm or the equivalent of 8 pixels.In this embodiment, imaging lens 420 is a telecentric lens with thefollowing specification: The field of view (FOV) is 2.25 inches, whichis wide enough to inspect two 27 mm parts (including a separation gap of0.125 inches). This corresponds to a maximum average magnification ofm=0.514. The minimum allowed average magnification is m=0.499 (3%decrease) and the allowed magnification variation along the central axis(for one lens) is +0.5%, but is preferred to be less than +/−0.2% (whichis equivalent +/−½ LSB of the range measurement). Compensation can beadded to the range calculations to reduce the affect of magnificationvariation if the variation is greater than +/−0.2%. The degree oftelecentricity (that is, how parallel is the central axis of theapertures across the FOV) must not change more than 0.01 degrees over an8 mil position change in the object plane. The position distortion mustnot exceed +/−1% of the FOV along the central axis. Ideally the positiondistortion should be less than +/−0.1%, but this can be obtained bysoftware compensation if the lens is unable to provide it. The maximumaperture opening must be at least f5.6 and preferably f4.0. The apertureshould be adjustable. (In another embodiment, Imaging lens 420 andProjection Lens 410 are each 35 mm f/4.0 lenses from Rodenstock Companyof Rockford, Ill.)

In this embodiment, the grating (i.e., projection pattern 412 of FIG.4A) is a sinusoidal line pattern, i.e., a repeating pattern of parallellines where the transmitted intensity varies as a sine wave as measuredalong a line perpendicular to the lines (Sine Patterns LLC of Penfield,N.Y. is the supplier of the Sine Pattern Grating, part number SF-3.0,used for one embodiment). The line pattern is oriented parallel to the 3rows of the CCD. The frequency of the sinusoid and the magnification ofthe projection lens is chosen so that one cycle along the verticalimaging axis 421 is 25.6 mils (0.0256 inches) long to give a rangeresolution of 0.1 mils (0.0001 inches).

In this embodiment, the magnification of projection lens 410 (see FIG.4A) is chosen so that one cycle along the vertical imaging axis is 25.6mils (0.0256 inches) long. The maximum aperture must be at least f4.0and possibly as large as f2.0. The aperture is not required to beadjustable. The magnification change across the central axis must be+/−0.5% or less and preferably less than +0.2%. The axis of the lens isrotated to provide an extended depth of focus of the line pattern in theimage axis 421. The rotation is such that the grating plane 413, imageaxis plane 451 and projection lens plane 411 tri-sect at line 419A perFIG. 4A.

In this embodiment, lens 414 (see FIG. 4A) is a condenser lens pair thatcollects light from the filament 416 and focuses the filament image 478onto the aperture of the projection lens 410. The aperture size shouldbe at least f1.0. In this embodiment, Condenser Lens 414 is a 35 mmf/1.0, part number 01CMPO13, from Melles Griot, Irvine, Calif.

In this embodiment, the recommended filament 416 is L7420 from GilwayTechnical Lamp, Woburn, Mass. The size of filament 416 (see FIG. 4A) is11.8×4.6 mm and the power is 400 watts. Other filaments with a similarpower rating can be substituted.

In this embodiment, mirror 408 (see FIG. 4A) is a concave sphericalmirror that has a radius equal to its distance from the filament. Thepurpose of spherical mirror 408 is to reflect light to the condenserlens 414. In this embodiment, a Concave Mirror, part number P43,464 fromEdmund Scientific, Barrington, N.J., is used. Since filament 416 blocksthe direct path, consideration is given to creating a virtual image 477of the filament 416 adjacent to the real filament.

In this embodiment, a reflecting infrared (IR) filter 450 (see FIG. 4D),e.g., IR Cut Filter, part number P43,452 from Edmund Scientific,Barrington, N.J., is used between the filament 416 and the condenserlens 414 is used to limit infrared (IR) going to the CCD because the CCDhas a poor MTF response in the IR range, and to reduce sphericalaberrations in the optical path.

In this embodiment, focus adjustment is provided so that the optimalfocus of both optical paths 409 and 421 occurs at the object 99.

Pattern Projector 402 Having Scheimpflug's Condition for Z-DimensionPlane

FIG. 4A shows an isometric schematic view of one embodiment of amachine-vision head 401 for inspecting a device 99. In one embodiment,machine-vision head 401 includes a pattern projector 402 and an imager404 (see, e.g., FIG. 4D). Pattern projector 402 includes a light source418, projection pattern element 412, and pattern projector imagingelement 410, wherein light source 418 provides light propagatinggenerally along a projection optical axis 409. Projection optical axis409 intersects the device 99 when the machine-vision head 401 is inoperation, as device 99 travels along direction 90 under head 401. Inone embodiment, projection pattern element 412 is located so that theprojection optical axis 409 passes through the projection patternelement 412 at an orthogonal (90 degrees) angle 407. Pattern projectorimaging element 410 is also located so that the projection optical axis409 passes through the pattern projector imaging element 410. In oneembodiment, pattern projector imaging element 410 is implemented as alens.

In this entire patent application, the term “lens,” unless otherwiseindicated, is meant to include any type of imaging element, such as acompound or simple lens, or a hologram or diffraction pattern that actsas a lens; one such embodiment, the lens is a compound lens; in otherembodiments, a simple lens is used; and in yet other embodiments, ahologram is used.

Focus Lamp Image into Projection Lens and Focus Grid Image onto Device

Lens-to-device distance D₁ and lens-to-grid distance D₂ are configuredto focus an image of projection pattern element 412 at the surface ofdevice 99 that is being inspected. In one embodiment, condensing imagingelement 414 (in one such embodiment, this is a compound lens; inanother, it is a simple lens as shown) is provided, and lens-to-lensdistance D₃ and lens-to-light-source-distance D₄ are configured so asfocus an image of light source 418 onto pattern projector imagingelement 410, and in particular to enlarge an image 476 of filament 416(for incandescent light sources) or other light source (e.g., a xenonflashlamp, or xenon-metal-halide tube) so as to fill the diameter (i.e.,the effective aperture) of lens 410, in order to maximize the amount oflight projected to device 99.

Linear Light Source to Increase Efficiency

One aspect of the present invention is that a pattern of light isprojected onto a device being measured, and then imaged onto a lineardetector 423, e.g., a trilinear-array detector. The pattern of lightprovides different intensities at different heights (Z dimension) and/ordifferent points along the scan direction (Y dimension), in order togather information needed to determine the three-dimensional geometry ofthe device being measured. For example, a sine-wave pattern isprojected, and each point is measured at three different phases of asingle cycle of the sine pattern. The system is designed so that thefirst phase measurement is detected by a pixel on the first line of thelinear detector 423, the second phase measurement is detected by a pixelon the second line of the linear detector 423, and the third phasemeasurement is detected by a pixel on the third line of the lineardetector 423. For such a three-phase measurement, light from otherphases of the sine-wave light pattern is not needed, and should beblocked so reflections form that light do not interfere with the desiredmeasurement.

In one embodiment, light source 418 includes an elongated incandescentfilament 416 wherein the longitudinal axis 406 of filament 416 isperpendicular to projection optical axis 409 and parallel to the gridlines of projection pattern element 412. In one embodiment, a sphericalor cylindrical reflector 408 is provided to focus an image 477 offilament 416, wherein filament image 477 is adjacent to filament 416, inorder to maximize the usable light projected towards device 99. (SeeFIG. 4A, showing filament 416, the first filament image 476 focussed bylens 414 to fill the width (X-=dimension) of lens 410, the secondfilament image 477 of filament 416 as focussed by reflector 408 to beadjacent to filament 416, and the third filament image 478—being animage of second filament image 477—focussed by lens 414 to fill thewidth (X-dimension) of lens 410.)

In one embodiment, a mask 442 is provided to reduce light that does notcontribute to the imaging function (i.e., light that, although perhapsproperly imaged from pattern 412 to object 99, might reflect off ofsurfaces that are not meant to be in the field of view of the imagingcomponents). In one such embodiment, mask 442 has a elongated aperture443 located adjacent to or in close proximity to projection patternelement 412, and wherein the long axis of aperture 443 is parallel tothe “lines” of the pattern of projection pattern element 412. In onesuch embodiment, projection pattern element 412 includes a pattern 472that transmits light having an intensity that varies in a sinusoidalpattern in the Z dimension (and in the Y dimension), but that isconstant across its width in the X dimension. That is, pattern 472varies substantially as a sine wave 474 along line 475 within element412, but is substantially constant along lines 473 that areperpendicular to line 475 (see FIG. 4E). Thus light 499 is imparted witha spatial-modulation pattern 494 that varies as a sine wave in intensityin one transverse direction (parallel to line 475), and that issubstantially constant along lines parallel to lines 473 that areperpendicular to line 475 (see FIG. 4E). It is this pattern that allowsmeasurements (e.g., three measurements at three different phases ofpattern 494) to determine 3D geometric measurements of part 99.

The device-to-imaging-lens distance D₅ andimaging-lens-to-trilinear-element distance D₆ are configured so as focusan image of device 99 onto trilinear array 423.

Imager 404 has a reception optical axis 421, the reception optical axis421 intersecting the device 99 when the machine-vision head 401 is inoperation. Imager 404 is maintained in a substantially fixedrelationship to the pattern projector 402. In one embodiment, imager 422includes three lines of semiconductor imaging pixels 423 (also calledtrilinear array 423).

Scheimpflug's condition is where the plane of the object, the plane ofthe lens, and the plane of the image all intersect at a single line.When Scheimpflug's condition is satisfied, an optimal focus of the imageof the object onto the image plane is achieved. Thus, wherein the imageplane is tilted to the plane of the lens (thus these two planesintersect at a line), then the plane of the object should be tilted soas to intersect the other two planes at the same line.

In one embodiment, the projection pattern element 412, the patternprojector imaging element 410, and a third plane 451 substantiallysatisfy Scheimpflug's condition. In one such embodiment, the receptionoptical axis 421 and the center line of trilinear array 423 both lie inthe third plane 451. When used to scan according to the presentinvention, the image formed on trilinear array 423 represents a verynarrow elongated area of device 99, i.e., narrow in the direction 90(dimension Y) and much longer in the direction perpendicular to thepaper of FIG. 4A (dimension X). However, as the height of features suchas solder balls 97 or substrate 98 varies, there is more distance inthis direction (dimension Z) that must be accommodated (i.e., perhapsseveral Moiré fringes in the Z dimension when measuring the height of apin on a pin-grid array device), or there is a desire for increasedaccuracy of measurement in the Z dimension. Thus, in this embodiment,the third plane 451 is in the X-Z dimension, and Scheimpflug's conditionis satisfied by an intersection of plane 411 (the plane that containsthe diameters of lens 410), plane 413 (the plane that contains thediameters of projection pattern 412) and a third plane at line 419A, asshown in FIG. 4A. In one such embodiment, the reception optical axis 421lies within the third plane 451 or is substantially parallel to thethird plane 451. In another such embodiment, the longitudinal axis ofthe center line of trilinear array 423 also lies within the third plane451 or is substantially parallel to the third plane, as shown in FIG.4A. Such an arrangement provides for an increased depth of focus in theZ-dimension for projected pattern 499.

FIG. 4C shows a cross-section view of yet another embodiment of amachine-vision head 401 of a machine-vision system 400C. A plane 91 liesacross the surface of device 99 being measured (the edge of plane 91,parallel to and at or near the surface of device 99, is shown in FIG.4C). Machine-vision system 400C is otherwise the same as system 400A ofFIG. 4A described above, however, Scheimpflug's condition is satisfiedfor the pattern plane 413, lens plane 411, and plane 91 which intersectat line 419C which extend in the Y dimension (perpendicular to the sheetat the point marked 419C on FIG. 4C).

FIG. 4D shows still another embodiment of a machine-vision head 401D.The machine-vision system 400D is otherwise the same as system 400Ashown in FIG. 4A, but with the addition of a non-patterned light source403 (such as the ring-light shown, or other source of general,non-patterned light 494), and IR filter 450. Non-patterned light source403 is non-patterned in the sense that the non-patterned light 494 has asubstantially uniform intensity in the X, Y, and Z dimensions at theportion of device 99 that is imaged onto detector 423 (in contrast topatterned light 499 which varies in the Y and Z dimensions). Oneembodiment includes a diffuser 493 which reduces specular reflectionsfrom, e.g., the individual LED light sources, of non-patterned lightsource 403. In one embodiment, a scan is performed by time-interleavingflashes from pattern projector 402 with flashes from non-patterned lightsource 403, each of which is detected by imager 404. The received imagesignals 405 are then demultiplexed, such that 2D images (i.e., theintensity of reflected light from each X and Y point) are derived fromthe non-patterned light 494, and 3D images (i.e., including thecalculated Z height of each X and Y point) are derived from thepatterned light 499. The projection pattern element 412′ shown in FIG.4D represents a square-wave pattern, but is implemented as a sine-wavepattern in other embodiments.

FIG. 4E shows a projection pattern element 412 having a density pattern472 that is a sine-wave in one direction, and a constant density in theperpendicular direction of the plane of the element. In such anembodiment, pattern projector 402 provides a projected pattern 499 whoseintensity along a line segment (e.g., a line segment along receptionoptical axis 421 at and near device 99) varies as a sine wave. In oneembodiment, the projected pattern has a sharp focus at a plane 451containing the line reception optical axis 421 and the center line ofthe three lines of optical detector pixels of trilinear array 423.

The projection pattern element 412′ of another embodiment shown in FIG.4F represents a square-wave pattern. In one such embodiment, patternprojector 402 provides a projected pattern 499 whose intensity along aline segment (i.e., a line segment very near pattern projector 402)varies as a series of pulses. In one such embodiment, the series ofpulses is substantially a square wave, i.e., the pattern is a series ofparallel of unilluminated and illuminated stripes (colloquially,“black-and-white” stripes). In one such embodiment, these square wavesare of such a fine pitch that the projected pattern 499 “blurs” or“diffracts” into substantially a sine-wave pattern of projected light,as measured on a line segment in the Z dimension (and on a line segmentin the Y dimension) at device 99. Projection pattern element 412′replaces projection pattern element 412 of FIG. 4A in some embodiments.

The projection pattern element 412″ of two other embodiments shown inFIGS. 4G and 4H represent an interleaved square-wave pattern. In FIG.4G, two glass plates are separated by a space (e.g., by air) and eachhas a parallel pattern on a single face, while in FIG. 4H, a singleplate having a parallel stripe pattern on one face and a complementaryparallel stripe pattern on the opposite face. In one such embodiment,the series of pulses is substantially a square wave, i.e., the patternis a series of parallel of opaque and transparent stripes (colloquially,“black-and-white” stripes), interleaved with another complementarypattern of parallel of opaque and transparent stripes that is spaced inthe direction of light propagation 409. In one such embodiment, thesesquare waves are of such a fine pitch that the projected pattern 499“blurs” or “diffracts” around the complementary edges into substantiallya sine-wave pattern of projected light, as measured on a line segment inthe Z dimension (and on a line segment in the Y dimension) at device 99.Projection pattern element 412″ replaces projection pattern element 412of FIG. 4A in some embodiments.

Movable Slit Aperture to Accommodate Different Heights of Devices

FIG. 4I shows yet another embodiment of a machine-vision head 401. FIG.4I shows a system 4001 that is otherwise the same as system 400A of FIG.4A, but with mask 442 having a slit aperture 443 which can be adjustedusing linear actuator 450. Linear actuator 450 moves only the positionof the aperture 443; it does not move projection pattern 412. In typicaluses of the present invention, devices 99 are placed into trays forinspection. It is desirable to inspect devices that differ in height (ortrays that are thicker or thinner, and thus present devices at differentheights relative to the tray), and linear actuator 450 allows movingaperture 443 so the light projects onto the portion of the device to bemeasured. In one embodiment, the computer control system dynamicallyadjusts linear actuator 450 so the light passed by mask 442 is projectedonly on the portion of device 99 that is imaged onto detector 423, inorder to keep the scan on the height of interest. In other embodiments,a static adjustment of actuator 450 is made to set the projector for theexpected height of the tray or devices.

FIG. 4B shows another embodiment, wherein the projection pattern element412 and the pattern projector imaging element 410 are combined andimplemented as a single hologram 430 or other suitable diffractionpattern (i.e., see the cross-section view of FIG. 4B wherein projectionpattern element 410 and pattern projector imaging element 412 arereplaced by hologram 430). Hologram 430 is fabricated using methods wellknown in the art (such as, for example, by forming a hologram of aprojection pattern element 412 and pattern projector imaging element 410and, or by forming a hologram of a hologram of projection patternelement 412 and pattern projector imaging element 410), and representsthe interference of a reference beam with light as modified byprojection pattern element 412 and pattern projector imaging element410. Then, when illuminated by a suitable reference beam 431 (such as anexpanded collimated laser beam), the desired projected Moiré lightpattern 499 is produced, having a focus range that is extended in the Zdimension at device 99. One advantage of such an arrangement is that thecoherent light of the laser light source is more sharply focussed,particularly when using a hologram focusing element 430.

In one such embodiment, the reception optical axis 421 lies within thethird plane 451 or is substantially parallel to the third plane 451,which substantially satisfies Scheimpflug's condition when combined withhologram 430 (which has a diffraction pattern representing or equivalentto projection pattern element 412 and pattern projector imaging element410 with their planes 413 and 411 respectively, at an angle to oneanother.

Interferometric Measurement at 45-Degree Angle to Minimize SpecularReflection

Shiny objects, such as solder ball connectors on ball-grid array (BGA)devices or pins of pin-grid-array (PGA) devices, present challenges foroptical measurement, because of specular reflections. Further, such pinsor solder balls present challenges due to shadowing of the relativelytall connector which obscures features “behind” the connector.

FIG. 5A shows one embodiment of the present invention that providespattern projector 402 at a forty-five (45)-degree angle 510 to thedirection of motion 90 (other embodiments use an angle α 510 (see FIG.5) of between about 45 degrees and about 50 degrees). FIG. 5A showsregion 520 of a solder ball 97 that is both lit by pattern light 499 andimagable by imager 404, region 522 that is not lit by pattern light 499but would have been imagable by imager 404 (“in the dark”), region 524that is lit by pattern light 499 but not imagable by imager 404 (“underthe ball”), and region 526 that is neither lit by pattern light 499 norimagable by imager 404 (“under the ball and in the dark”). Since theangle of incidence equals the angle of reflection, the center of thespecular reflection 528 on a spherical solder ball will be about 23degrees from the top of the ball, thus away from the top of the ballwhere the measurements most critical to accurately measuring the heightand position of the top of the ball are made. Region 528 represents thatportion of ball 97 where a specular reflection will saturate thedetector 423 in imager 404, preventing any accurate height measurement(most of the rest of region 520 provides accurate measurement fromdiffuse reflection). Further, the amount of shadowing (i.e., the area ofregion 522) is reduced as compared to using a smaller angle 510 relativeto the direction of motion.

In FIG. 5A, ray 530 and ray 531 represent light that reflects fromneighboring balls 97 onto the ball 97 being measured. Regions 529represent portions of the center ball 97 which have specular reflectedlight from rays 530 and 531 reflected from neighboring balls in the Ydirection (all such light which is reflected from neighboring featuresis “noise” light that reduces the accuracy of height measurements). Byproviding a slit mask 442 (i.e., a mask having an aperture that extendsin the X-dimension but is much narrower in the Y-dimension; see FIG.4A), the source light for rays 530 and 531 is blocked or reduced,preventing or reducing the reflections from features in the Y dimensionfrom reaching the portion of device 99 being measured.

In other embodiments, mask 442 is designed to reduce unwantedreflections of light from features displaced in the X-dimension as well.For example, color-filter stripes (e.g., a repeating pattern of red andblue stripes) that are perpendicular to the long axis of the aperture443 are used in one embodiment to reduce noise reflections from theX-dimension.

FIG. 5B is a representation of light gathered by a non-telecentric lens420. In detecting the light at the three measurement points 571, 572,and 573, the projected light 499 is gathered from cones that center onthe aperture of lens 420. This can cause measurement errors due togathering light at different reflection angles for the threemeasurements of the ball 97 as it moves from point 571 to 572 to 573. Incontrast, FIG. 5C is a representation of light gathered by a telecentriclens 420. A telecentric lens (or other similar imaging element) 420gathers light as if from an infinite number of apertures, and thus thelight gathered is from the same reflection angle (i.e., a vertical anglefor this embodiment) for the three measurements of the ball 97 as itmoves from point 571 to 572 to 573 in FIG. 5C. Thus a telecentricimaging lens 420 provides increased measurement accuracy for someembodiments.

In one embodiment, projection pattern element 412 is a Sine PatternGrating, model SF-3.0, available from Sine Patterns LTD, Penfield, N.Y.;Imaging lens 420 and Projection Lens 410 are 35 mm f/4.0, available fromRodenstock, Rockford, Ill.; Condenser Lens 414 is 35 mm f/1.0, model01CMPW13, available from Melles Griot, Irvine, Cal.; light source 481 isa Halogen Lamp, 400 watt, model L7420, available from Gilway TechnicalLamp, Woburn, Mass.; IR filter 450 is an IR Cut Filter, model P43,452available from Edmund Scientific, Barrington, N.J.; reflector 408 is aConcave Mirror, model P43,464, available from Edmund Scientific,Barrington, N.J.; trilinear array detector 423 is a Tri-Linear CCD,model KLI-2113, available from Eastman Kodak Co, Rochester, N.Y.

FIG. 6 shows machine-vision system 600 that represents anotherembodiment of the present invention having more than one projector 602in a scanning head 601. In one embodiment, gantry 610 (not shown toscale) is used to scan head 601 in the Y direction during measurement,but is also used to move head 601 in the X direction for successive scanoperations, and to move head 601 in the Z direction to adjust forthicker or thinner devices, for example. In the embodiment shown,projector 602A projects a light pattern 499A that includes, for example,phases (i.e., phase angles of a sine wave) α1, α2, α3, α4, and α5, wherethe line for α1 represents a plane perpendicular to the drawing sheethaving a constant light intensity representing α1, the line for α2represents a plane perpendicular to the drawing sheet having a constantlight intensity representing α2, etc. (Actually, a sine wave patternhaving an infinite number of intermediate phases is projected, five ofwhich are shown in this illustration.) Similarly, projector 602Bprojects a light pattern 499B. In this embodiment, the first projector602A and the second projector 602B are oriented substantially oppositefrom one another relative to the imager 604, and their patterns of light499A and 499B provide grid lines that project parallel to one anotherwhen projected onto a horizontal (x-y) or vertical plane (y-z). Scanningis done by moving head 601 in the Y direction. In this embodiment, theoptical axis of sensor 604 is a right angles to the direction ofscanning 90 (the Y direction), and each point of device 99 is measuredat Y1 at a first point in time t1, at Y2 at a second point in time t2,and at Y3 at a third point in time t3. (For example, when a 3-by-2048trilinear detector 423 is used, the 50th pixel of the first line, the50th pixel of the second line, and the 50th pixel of the third line areused for the first, second and third measurement respectively of eachpoint 50/2048^(ths) of the way across the scan path, and a correspondingpixel of each of the three lines gathers information for the otherpoints of the scan.) In one embodiment, projector 602A and projector602B are oriented one-hundred eighty degrees from one another relativeto sensor 604, and are strobed alternately (i.e., flashes of light fromeach projector are interleaved) (one strobe for each triple scan-lineoperation), and a single sensor 604 is used to gather the imageinformation. In such an embodiment, an LED, laser, xenon, arclamp, orother strobable or shutterable light source is used. This allowsgathering of light from all sides of a feature (e.g., relative to FIG.5A described above, the unlit region 522 from projector 602A would be inthe illuminated region 520 of projector 602B, and vice versa) withlittle or no shadowing relative to the projectors. In one suchembodiment, sensor 604 is at a 90-degree angle to the Y dimension,projector 602A is at a 45-degree angle, and projector 602B is at a135-degree angle (i.e., at a 45-degree angle on the other side).

In another such embodiment, a complete scan in the +Y direction, forexample, with only projector 602A providing illumination, would befollowed by a complete scan in the −Y direction, for example, with onlyprojector 602B providing illumination. Thus features that were shadowedduring one scan would be illuminated during the other scan. In such anembodiment, an incandescent (non-strobed) or strobable light source isused.

In yet another embodiment, the light from projector 602A is filtered orfrom a monochromatic light source (e.g., red color), and is at adifferent color (frequency) than the light from projector 602B which isfiltered or from a monochromatic light source at a different color(e.g., blue light). In this embodiment, sensor 604 is provided with acolor separator (e.g., a dichromatic beamsplitter), and one trilineardetector 423 for each color. (See FIGS. 9A, 9B, and 9C below for furtherdescription.)

In still another embodiment, the light from projector 602A is polarized,and is at a different polarization than the light from projector 602Bwhich is also polarized. In this embodiment, sensor 604 is provided witha polarization separator (e.g., a polarized beam splitter), and onetrilinear detector 423 for each polarization.

In the embodiment shown in FIG. 6, the optical axis of sensor 604 is atright angles to the scan direction 90, and thus each point is measuredat the same Y1, Y2, and Y3 positions by sensor lines S1, S2, and S3,respectively, regardless of height (i.e., the Z dimension). Thus, the Zdimension of a point having height H1 would be measured relative toprojector 602B as a function of (α1 at time t1 measured by sensor S1, a2at time t2 measured by sensor S2, and a3 at time t3 measured by sensorS3) measured from projection pattern 499B. Similarly, the Z dimension ofa point having height H2 would be measured relative to projector 602B asa function of (α2 at time t1 measured by sensor S1, α3 at time t2measured by sensor S2, and α4 at time t3 measured by sensor S3) measuredfrom projection pattern 499B. In one embodiment, one or more linearencoders are used to determine the exact position of head 601 in the Xand Y directions at each point in time, providing X and Y coordinatedfor each point measured. The Z dimension is derived from the three phasemeasurements made for each point, since each point's height has a uniqueintersection with the various phases projected. In one embodiment, theintensity for each point is also derived from these measurements. The Zdimension derived from the measurements made from the light projectionpattern 499A (denoted z′) is also derived using similar measurements.Table 1 below illustrates the derivation of the height dimension (z forthe measurements from projector 602B, and z′ for the measurements fromprojector 602A) for three heights H1, H2, and H3.

TABLE 1 Measurement using Measurement using Height projector 602Aprojector 602B H1 z′ = f((α3, t1), (α2, t2), z = f((α1, t1), (α2, t2),(α3, t3)) (α1, t3)) H2 z′ = f((α4, t1), (α3, t2), z = f((α2, t1), (α3,t2), (α4, t3)) (α2, t3)) H3 z′ = f((α5, t1), (α4, t2), z = f((α3, t1),(α4, t2), (α5, t3)) (α3, t3))

FIG. 7 shows machine-vision system 700 that represents anotherembodiment of the present invention having more than one imager 704 in ascanning head 701. In one such embodiment, projector 702 is at a90-degree angle to the Y dimension, sensor 704A is at a 45-degree angle,and sensor 704B is at a 135-degree angle (i.e., at a 45-degree angle onthe other side). In contrast to the system 600 of FIG. 6 in which anyfeature is measured at the same sensor pixel at a given time,independent of its height, the system 700 of FIG. 7 must take intoaccount the fact that a feature will shift in time (i.e., in the Ydirection) as it varies in height. For example, if a point has heightH1, it will be measured at times t0, t1, and t2 by sensors s1, s2, ands3 respectively, but would be measured at times t2, t3, and t4respectively if it had height H3. Thus the derived height information isused to correct for Y-direction displacement caused by not having theoptical axis of sensors 704A and 704B not at right angles to thedirection of scanning 90. Further, the reader will note that the timepoints used to measure a feature on device 99 will shift as the heightsshift beyond the amount which can be measured by a particular pixel at aparticular time. However, because heights often change gradually, allheights can generally be derived accurately (e.g., for Projector 602A,if the height difference between H1 and H2 is h, then for heightsH1+/−h/2, the heights are derived using the measurements from times t0,t1, and t2; for heights H2+/−h/2, the heights are derived using themeasurements from times t1, t2, and t3; and for heights H3+/−h/2, theheights are derived using the measurements from times t2, t3, and t4).Since measurements are repeated for hundreds or thousands of time pointst, and are stored in a data structure representing every time point, thedata needed to correct for the time shift due to height changes isavailable and used to correct the Y values according to the derived Zvalue of each point. Table 2 shows the relationship of these values.

TABLE 2 Measurement using Measurement using Height projector 602Aprojector 602B H1 z′ = f((α1, t0), (α2, t1), z = f((α3, t2), (α4, t3),(α5, t4)) (α3, t2)) H2 z′ = f((α2, t1), (α3, t2), z = f((α2, t1), (α3,t2), (α4, t3)) (α4, t3)) H3 z′ = f((α3, t2), (α4, t3), z = f((α1, t0),(α2, t1), (α3, t2)) (α5, t4))

FIG. 8 shows machine-vision system 800 that represents anotherembodiment of the present invention having more than one projector 802and more than one imager in a scanning head 801. In this embodiment,projector 802A has a projection optical axis at a compound miter angleand projects sine-wave pattern 499A, wherein the lines of constantintensity are at a 45-degree angle β_(A) to the direction of scanning 90(rather that the 90-degree angle used in the embodiment of FIG. 4A).Similarly, projector 802B has a projection optical axis at a compoundmiter angle and projects sine-wave pattern 499B, wherein its lines ofconstant intensity are at a 45-degree angle β_(B) to the direction ofscanning 90, and at a 90-degree angle to the lines of constant intensityof sine-wave pattern 499A. Beamsplitter cube 820 is used to allowimaging of pattern 499A onto detector 423A, and to allow imaging ofpattern 499B onto detector 423B.

In this embodiment, the optical axis of sensor 804 is a right angles tothe direction of scanning 90 (the Y direction), however the long axis ofeach detector array 423 is at a 45-degree angle to the direction ofscanning 90, and each point of device 99 is measured at Y1 at a firstpoint in time, at Y2 at a second point in time, and at Y3 at a thirdpoint in time. (For example, when a 3-by-2048 trilinear detector 423having an eight-pixel spacing between lines is used, the 50th pixel ofthe first line, the 58th pixel of the second line, and the 66th pixel ofthe third line are used for the first, second and third measurementrespectively of each point 50/2048^(ths) of the way across the scanpath, and a corresponding pixel of each of the three lines gathersinformation for the other points of the scan.)

As described in more detail above, in one embodiment projector 802A andprojector 802B are strobed alternately (i.e., flashes of light from eachprojector are interleaved) (one strobe for each triple scan-lineoperation), and sensor 804 having detectors 423A (synchronized toprojector 802A) and 423B (synchronized to projector 802B) is used togather the image information. This reduces the area that is shadowed. Inone such embodiment, sensor 604 is at a 90-degree angle to the Ydimension, projector 802A is at a 45-degree angle vertically and a45-degree angle horizontally, and projector 802B is at a 45-degree anglevertically and a 45-degree angle horizontally (i.e., at a 45-degreeangle on the other side).

In another such embodiment, a complete scan in the +Y direction, forexample, with only projector 802A providing illumination, would befollowed by a complete scan in the −Y direction, for example, with onlyprojector 802B providing illumination. Thus features that were shadowedduring one scan would be illuminated during the other scan. In such anembodiment, an incandescent (non-strobed) or strobable light source isused for each projector.

In yet another embodiment, the light from projector 802A is filtered orfrom a monochromatic light source (e.g., red color), and is at adifferent color (frequency) than the light from projector 802B which isfiltered or from a monochromatic light source at a different color(e.g., blue light). In this embodiment, sensor 804 is provided with acolor separator (e.g., a dichromatic beam splitter 820), and onetrilinear detector 423 for each color. (See FIGS. 9A, 9B, and 9C belowfor further description.)

In still another embodiment, the light from projector 802A is polarized,and is at a different polarization than the light from projector 802Bwhich is also polarized. In this embodiment, sensor 804 is provided witha polarization separator (e.g., a polarized beam splitter 820), and onetrilinear detector 423 for each polarization.

In one embodiment, gantry 610 (not shown to scale) is used to scan head801 in the Y direction during measurement, but is also used to move head601 in the X direction for successive scan operations, and to move head601 in the Z direction to adjust for thicker or thinner devices, forexample.

FIG. 9A shows an isometric view of sensor 904A having a beamsplitter820A such as is used in one embodiment of system 400A of FIG. 4A. Inthis embodiment, trilinear detector 423A is superimposed optically ontotrilinear detector 423A (i.e., corresponding pixels of each trilineardetector receive light from identical points of device 99). In one suchembodiment, the direction of scanning (the Y dimension) is diagonallyacross the top face 821 of beamsplitter 820, and thus at a right angleto the long axis of detector 423A. This provides an image that alsomoves at a right angle to the long axis of detector 423B.

Two Discrete Imagers (Dichroic Beam Splitter) Interleaved orSimultaneous

In one such embodiment, the trilinear detectors 423A and 423B areclocked so that they provide interleaved line images, thus providing adoubling of the speed achievable when imaging is otherwise limited bythe clock speed of a single trilinear array detector. By aligning thetwo detectors with one another, and interleaving their respectiveacquisition of line-scan data, every other scan line (e.g., theeven-numbered lines) comes from detector 423A, while the other lines(e.g., the odd-numbered lines) comes from detector 423B. In one suchembodiment, the projector light source 402 is strobed, in order toreduce smearing of the image (i.e., capturing data from an region thatis extended in the Y dimension due to the increased scanning speed).

Two Discrete Interleaved Imagers (50-50 Beam Splitter)

In one such embodiment, beamsplitter 820 provides a 50%-50% beamsplitting function, thus providing equal illumination on trilineardetectors 423A and 423B. In one such embodiment, projector 404 isstrobed at a different intensity for trilinear detector 423A than fortrilinear detector 423B, in order to achieve an improved light-intensitydynamic range (i.e., a dimly lighted view and a brightly lighted view ofthe same device). This is because highly reflective points on device 99can saturate the detector with a given illumination, while slightlyreflective points will be too dark. By providing high illumination onone strobe, the dim points are accurately imaged, and low illuminationon the interleaved strobe, the bright points are accurately imaged. Inanother such embodiment, the time for which trilinear detector 423Acollects light is kept shorter (“shuttered” to a faster time) than fortrilinear detector 423B, in order to achieve the same results.

In another such embodiment, beamsplitter 820 provides avariable-intensity (e.g., a 20%-80%) beam-splitting function. Thisprovides an improved light-intensity dynamic range (i.e., a dimlylighted view and a brightly lighted view of the same points) such that,e.g., trilinear detector 423A can receive four times as much lightduring a given time period than does trilinear detector 423B (as well asperhaps achieving a speed improvement).

In still another embodiment, beamsplitter 820 provides a variable-color(e.g., a dichroic beamsplitter such as a red-blue color separating)beam-splitting function. In one such system, (such as system 600 of FIG.6), one projector (e.g., projector 602A) projects a sine-wave stripedpattern of red light, and the other projector (e.g., projector 602B)projects a sine-wave striped pattern of blue light onto the same regionof device 99. (Were the same color projected simultaneously, the twopatterns might interfere with each other's measurement operations.) Thedichroic beamsplitter 820A separates the pattern from projector 602Afrom that of projector 602B allowing simultaneous measurements to thedifferent colors by separate detectors 423A and 423B.

In yet other embodiments, a beamsplitter that provides three or moreseparations (e.g., one that provides splitting of three or more colors,or three or more intensities, etc.) is used, with a separate detector423 for each light path provided.

Two Interleaved Imagers (Perpendicular Pattern Projector, Imager at 45Degrees)

FIG. 9B shows an isometric view of sensor 904B having a beamsplitter820B such as is used in one embodiment of system 800 of FIG. 8. In thisembodiment, the long axis of long axis of detector 423A is at a rightangle to the long axis of detector 423B as viewed from optical axis 421.In this embodiment, the direction of scanning (the Y dimension) isparallel to an edge (e.g., edge 822 of the top face 821 of beamsplitter820B), and thus at a 45-degree angle to the long axis of detector 423A.This provides an image that also moves at a 45-degree angle to the longaxis of detector 423B. As with FIG. 9A, this arrangement of positioningthe trilinear detector 423 diagonally provides a more efficient use ofbeamsplitter material (e.g., a smaller cube can handle a given lengthtrilinear detector 423. As with FIG. 9A, the trilinear detectors 423Aand 423B are mounted next to the respective faces 821 and 823 ofbeamsplitter 820B, however, in other embodiments, the detectors arespaced away from the respective faces 821 and 823 of beamsplitter 820B.

FIG. 9C shows an isometric view of sensor 904C having a beamsplitter820C. In this embodiment, the long axis of long axis of detector 423A isparallel to the long axis of detector 423B as viewed from optical axis421, and both are substantially parallel to an edge 824 of thebeamsplitter cube. In this embodiment, the direction of scanning (the Ydimension) is parallel to an edge (e.g., edge 822 of the top face 821 ofbeamsplitter 820B), and thus at a 90-degree angle to the long axis ofdetector 423A. This provides an image that also moves at a 90-degreeangle to the long axis of detector 423B.

In variations of each of the embodiments of FIGS. 9A, 9B, and 9C, thesplitting function can be done by any suitable combination of splittingmaterial and arrangement, such as multiple dielectric layers, prisms,polarizers (Brewster's windows), etc., such as are well know in the art,and each are contemplated in different embodiments of the presentinvention.

Four Interleaved Imagers (2 Beam Splitters and Perpendicular PatternProjector)

Further, various combinations are contemplated of the beam splitters ofFIGS. 9A, 9B, and 9C with the systems 400, 600, 700, and 800 of FIGS. 4,6, 7, and 8. For example, the sensor 804A and 804B of FIG. 8 are eachimplemented as sensor 904A in one embodiment, in order to acquire scandata with four interleaved sensors (thus acquiring data at up to fourtime the scan speed as is possible using a single sensor 404 and singleprojector 402.

Another aspect of the present invention provides a method for highspeed, scanning phase measurement of a device at a machine-visionstation to acquire physical information associated with the device. Themethod includes the steps of: (1) projecting light generally along aprojection optical axis 409, the projection optical axis intersectingthe device; (2) spatially modulating the light with a Moire patternlocated so that the projection optical axis passes through the Moirepattern; and imaging the spatially modulated light onto the device; and(3) receiving light reflected from the device along a reception opticalaxis with an imager maintained in a substantially fixed relationship tothe projected spatially modulated light, the imager including threelines of semiconductor imaging pixels, the reception optical axisintersecting the device; (4) generating data representing acquiredthree-dimensional device geometry data regarding the device from signalsfrom the imager; (5) wherein the step of spatially modulating and thestep of imaging the spatially modulated light provide a light patternthat is focused along a region of a third plane, wherein one of thethree lines of semiconductor imaging pixels lies substantially withinthe third plane.

In one such embodiment, the step of spatially modulating and the step ofimaging the spatially modulated light, and a third plane substantiallysatisfy Scheimpflug's condition, and wherein the reception optical axislies within the third plane or is substantially parallel to the thirdplane.

In another such embodiment, the step of spatially modulating provides aprojected pattern whose intensity along a line segment varies as a sinewave.

Another such embodiment further includes the steps of: (6) comparing theacquired three-dimensional device geometry data with an intendedpredetermined geometry to produce a signal indicative of any devicegeometry departure of an actual device geometry from the intendedpredetermined geometry; and (7) controlling a manufacturing operation ofthe device to compensate for said device geometry departure.

Telecentric Imaging

Another important aspect of the present invention provides telecentricimaging in imager 402. As above, machine-vision head 401 is provided forinspecting a device. In this embodiment, imager 402 includes atelecentric imaging element 420 that focuses an image of the device ontothe three lines of semiconductor imaging pixels. A telecentric lens 420provides an imaging system that provides increased accuracy ofmagnification across the width of the object and image field,particularly for imaging pixels that are radially displaced from thereception optical axis 421, i.e., displaced in the X dimension for FIG.4A. In particular, the combination of substantially satisfyingScheimpflug's condition (for the plane 413 of the pattern element 412,the plane 411 of the projection imaging element 412, and the plane 451which extends in the Z dimension at the object or device being imaged 99and containing optical axis 421 and the centerline axis of the trilineararray 423) in order to create a light pattern that is highly accurate inthe Z-dimension, and the use of a telecentric imaging lens for lens 420which provides increased accuracy across the field of view, providesgreatly improved accuracy of three-dimensional data derived from theimaging operation. In various embodiments described elsewhere in thisdescription, a telecentric imaging element is used for imager imagingelement 420. In some embodiments, a hologram 432 (see FIG. 4D) providingimaging function equivalent to a telecentric lens is provided for imagerimaging element 420. According to the present invention, telecentricimaging provides the advantage of maintaining better accuracy ofmagnification and sharper focus of features on device 99 that vary inheight (i.e., the Z-dimension), particularly for features displaced fromthe reception optical axis 421 in the X dimension (i.e., features thatwill be imaged onto pixels displaced from the center of the threeimaging lines of trilinear array 423.) This provides a better accuracyof height measurements at the sides (i.e., portions displaced from theoptical axis in the X dimension; see FIG. 4A) of a scan.

Telecentric Projecting

Another aspect of the present invention provides telecentric projectingin pattern projector 402. As above, machine-vision head 401 is providedfor inspecting a device. In this embodiment, imager 404 includes atelecentric imaging element 410 that focuses an image of the projectionpattern element 412 onto the plane containing receiver optical axis 421and the center line of the three lines of semiconductor imaging pixels.A telecentric lens used for pattern projector imaging element 410provides an imaging system that provides increased depth of focus,particularly for device features that are radially displaced from theprojection optical axis 409, i.e., displaced in the X dimension for FIG.4A. In various embodiments described elsewhere in this description, atelecentric imaging element is used for pattern projector imagingelement 410. In some embodiments, a hologram 430 (see FIG. 4D) providingimaging function equivalent to a telecentric lens is provided forpattern projector imaging element 410. According to the presentinvention, telecentric projecting provides the advantage of maintainingsharper focus of features on device 99 that vary in height (i.e., theZ-dimension), particularly for features displaced from the receptionoptical axis 421 in the X dimension (i.e., features that will be imagedonto pixels displaced from the center of the three imaging lines oftrilinear array 423.) This provides a better accuracy of heightmeasurements at the sides of a scan, as well as perhaps betterY-dimension measurements at the sides of the scan.

Color Filters, IR Blocking Filters, Monochromatic Light Sources

In one embodiment, a color filter 450 (see FIG. 4D) is placed betweenlight source 418 and condensing imaging element 414, in order to sharpenthe focus of light 498 and light pattern 499 (since it is easier andsimpler to focus monochromatic light). In one embodiment, color filter450 also blocks infra-red (IR) components of light 498, in order thatthese IR components do not get received by imaging pixels 422. Invarious embodiments, light source 418 is implemented as a halogenincandescent lamp, as a xenon flashtube, as a metal-halide arc lamp, asa laser, or as an array of LEDs (light-emitting diodes). In embodimentshaving substantially monochromatic light output, such as LED arrays, theneed for a color filter is reduced. However, in some such embodiments,LEDs can still have substantial infrared components to their colorspectrums, and in some such embodiments, an IR blocking filter is usedfor filter 450, in order to reduce the IR components of the light output498. When single-frequency coherent sources such as lasers are used forlight source 418, the need for color filters is generally eliminated.

In other embodiments, a color filter 452 is used, and is placed in theoptical path of imager 404. In some embodiments, color filter 452includes an IR-blocking function to block IR components of the receivedlight. In some such embodiments, color filter 452 helps to removeunwanted ambient light, including IR components thereof.

Masks to Constrain Projected and/or Observed Illumination

In some embodiments, a mask 440 (see FIG. 4A) is used to block unwantedand stray light coming from sources other that the region of device 99that is intended to be imaged onto trilinear array 423. In one suchembodiment, mask 440 is painted or coated with a flat black surface, andhas a slit aperture 441 extending in the X dimension, but narrow in theY dimension. In one embodiment, slit 441 is configured with sharp (e.g.,knife) edges to reduce reflections of the edges of the slit. Slitaperture 441 is configured to block light from those projected linesthat fall to the sides (e.g., to the left or the right of receptionoptical axis 421 in FIG. 4A). Mask 440 is also configured so as to notblock any of the Moire-type stripes or lines that extend in the Zdimension. This is because the light of interest can extend for a largenumber of Moire fringes in the Z dimension in order to obtain heightinformation, and for a considerable distance on either side of thereception optical axis 421, in order that a wide region can be scannedin a single sweep but the light of interest in constrained to only theangle in the Y dimension that will properly image onto the three linesof the trilinear array 423. Thus all light in the Y-dimension that fallsoutside this narrow angle can and should be masked or blocked.

In general, a narrow aperture mask is not called for to block projectedpattern 499 since as many stripes in the vertical (Z dimension) aspossible are desired to obtain height information for an extended range,however in some embodiments a mask 442 having an aperture 443 is used toreduce stray light from pattern projector 402, and in particular, toreduce specular reflections on the solder ball being measured of lightreflected off surrounding balls (i.e., the mirror-like surface of theball being measured will generally reflect the images of all surroundingballs, and by limiting the light that is projected on those surroundingballs, the unwanted reflections from those balls are minimized). In suchembodiments, an aperture is configured having a length and width toaccommodate the desired width and height of the measurement desired.

In other embodiments, the entire system 400 is enclosed in alight-blocking enclosure or shroud 445 to block out substantially allambient light, and thus maximize signal-to-noise ratios of the acquiredimage data.

Sine-Wave Moire Pattern for Better Linearity and More Accurate Results

In some embodiments, a projection pattern element 412 having a sine-waveprofile is used (i.e., the stripes that extend in the X dimension with asubstantially constant density, vary as a sine wave in the directiontowards line 419A, See FIG. 4A). In various exemplary embodiments, thestripes have a pitch of three to ten stripes per millimeter. Sine-wavestripes give better linearity and more accurate results in someembodiments.

In other embodiments, a projection pattern element 412′ having asine-wave profile is used (i.e., the stripes that extend in the Xdimension with a substantially constant density, vary as a square wavein the direction towards line 429, See FIG. 4D). In various exemplaryembodiments, the stripes have a pitch of three to ten stripes permillimeter. Embodiments having square wave stripes can provide goodresults particularly at finer pitches, since the projected pattern willfocus to a pattern that has substantially a sine-wave profile atreception optical axis 421.

Offset Slit Pattern to Generate Sine Moire Pattern

In one embodiment of the present invention provides two stripedpatterns, wherein the plane of one set of stripes is parallel to andoffset from the plane of the other set of stripes. In one suchembodiment, the two striped patterns are square-wave, parallel opaquestripes that are on either side of a transparent separator. In one suchembodiment, no projection imaging element 410 is used, since theresulting light pattern provides the desired sine-wave pattern at themeasurement plane (the region near and at the device 99 of the planethat includes reception optical axis 412 and the center line of thetrilinear array 423).

Modular Machine-Vision System

FIG. 10 shows a modular machine-vision system 1000 of one embodiment ofthe present invention. In the embodiment shown, two cameras 1001A and1001B are 2D and 3D inspection cameras respectively, and acquire 2D and3D of the, for example, bottom surfaces of devices 99 in trays. Gantry1630A and gantry 1630B are controlled by computer 1010, and provide thescanning motion for cameras 1001A and 1001B. In one such embodiment,camera 1001A is a 3D inspection camera such as head 401 of FIG. 4A, andcamera 1001B is a conventional raster scan CCD imaging camera with astrobed ring-light illumination. Gantry 1630C and gantry 1630D arecontrolled by computer 1010, and provide the scanning motion for cameras1001C and 1001D. These separate gantries provide independent scanningmotions as may be needed for the 2D and 3D cameras, which may requiredifferent scanning speeds or x-y motions. In other embodiments, a singlegantry 1630 is used to control both cameras 1001A and 1001B, and/or asingle gantry 1630 is used to control both cameras 1001C and 1001D, insituations where a common motion to scan both cameras can be used.Elevators 1700A and/or 1700B are used to input trays of parts intosystem 1000 for inspection. Conveyor 1800A is used to hold traysstationary for inspection under cameras 1001A and 1001B, and to movetrays from inspection station A to inspection station B to flippingstation F. Conveyor 1800B is used to move trays from flipping station Fto inspection station C to inspection station D to picking station E andpicking station G, and to hold trays stationary for inspection undercameras 1001C and 1001D. One or more pickers 2000 are used to selectdevices from trays and to fill trays having all-good parts for output atelevator 1700G of station G, for example.

Computer 1010 for this embodiment includes keyboard input device 1016,display output device 1015, program media I/O device 1017 (e.g., adiskette drive), I/O signals 1011 used to control and receive input fromthe other components of system 1000, and I/O signals 1012 (e.g., alocal-area-network) used to control other steps in the manufacturingprocess and/or send and receive data useful for the manufacturingprocess or its control or monitoring. In one embodiment, program media1018 and/or network signals 1012 are used to input control programs tocomputer 1010.

In other embodiments, the functions of the stations described above arecombined or eliminated to provide a lower-cost system. In one suchembodiment, stations A, D, E, and G are eliminated, a single inputelevator (e.g., 1700B) is used, camera 1001B is a 2D/3D combined camerasuch as is shown in 401D of FIG. 4D, wherein alternated strobes ofpatterned light and non-patterned light are used to acquire 3D and 2D(respectively) line scan data. Similarly, a single output elevator(e.g., 1700C) is used, camera 1001C is a 2D/3D combined camera such asis shown in 401D of FIG. 4D. A flipper 1900 inverts the devices betweenstation B and C. One picker 2000 is used at station C to removedefective devices.

In yet other embodiments, stations A, B, and C have elevators 1700 thatare used to input trays of parts to be inspected; and stations D, E, andG have elevators 1700 that are used to output trays of inspected parts.Conveyor 1800A moves trays from any of the three input elevators 1700 tostations A and B for inspection (trays from elevator 1700C aretransferred from conveyor 1800B to conveyor 1800A at station F. Conveyor1800B transfers trays from/between the flipper, inspection stations Cand D, and picker stations E and G. Each of the six elevators 1700 areused to queue parts that have been loaded and are waiting to beinspected, or parts that have been inspected and are waiting to beunloaded. The loading and unloading of the elevators is performed byhuman users in one embodiment, and is performed by automated robots inother embodiments.

In still another embodiment, the elevators 1700 of FIG. 10 areeliminated, and trays of devices (or in other embodiments, devices inother containers, or devices attached to film strips, or even devicesnaked on a conveyor to be inspected) enter the system 1000 from the lefton a conveyor from the last manufacturing process. The trays then passsideways though one or more of the inspection stations A, B, C, and/orD, the flipper station F, and/or the pick-and-place stations E and/or G.In one such embodiment, the trays then pass to some other operation,such as packaging, assembly, or shipping, out the right side of system1000. Such inspection is sometimes called “in-line inspection.”

One important aspect of the present invention is to reduce the timeneeded to move trays of parts. To accomplish this, trays are preferablymoved along their shortest edge (i.e., the long edge of the trays arenext to one another, and the trays are moved in the directionperpendicular to this). Further, each station is as narrow as possibleand as close as possible to it neighbor stations, i.e., the elevators1700 and cameras 1001 have narrow edges and are mounted as close aspossible to one another, to minimize the distance between stations. Thecameras 1001 are scanned along the long dimension of the trays, tominimize the number of times the camera is stopped and moved to the nextscan path. Further, inspection starts on top of the first inputelevator, so that the trays need not be moved sideways from the firstelevator to the first inspection station which would take extra time.

FIG. 11 shows a computation and comparison system 1100 of one embodimentof the present invention. In one embodiment, computation and comparisonsystem 1100 is implemented as a software program and data structure thatoperates in computer 1010 of FIG. 10. Scan data from trilinear line-scandetector 423 is loaded into array 1110, with successive lines of digitaldata values from the first line going into subarray 1111, data from thesecond line going into subarray 1112, and data from the third line goinginto subarray 1113.

In one embodiment, a calculation is performed on a line-by-line basis tocorrect for intensity-level bias, drift or noise. For example, in oneembodiment, one or more pixels, e.g., at the beginning of a scan line,are kept in the dark (i.e., unilluminated by any image of the device 99)in order to detect a bias or drift in the value associated with “black.”This black-value bias is then subtracted from each data value for theline.

In one embodiment, the trilinear array 423 has three lines of 2048pixels each, with a pixel-to-pixel distance of 14 microns and aline-to-line distance of 112 microns (i.e., each photosite is squarewith a pixel-to-pixel center-to-center distance of 14 μm and theline-to-line center-to-center spacing is 112 μm or the equivalent of 8pixels). The camera 1001 is moved by one pixel distance between scans(in one embodiment, the scan movement is perpendicular to the long axisof the lines; in another embodiment, it is at a 45-degree angle). Thus aparticular point on device 99 will be measured by a particular pixel ofline 1 of trilinear array 423 going into subarray 1111, then eight scanslater that same particular point on device 99 will be measured by thecorresponding pixel of line 2 of trilinear array 423, data from thesecond line going into subarray 1112, and then eight scans later thatsame particular point on device 99 will be measured by the correspondingpixel of line 3 of trilinear array 423 and data from the third linegoing into subarray 1113. Thus for a given Y value there are three linesof data representing light intensity of the intersection of the lightpattern 499 with device 99: the index i points to the data in array1111, the index i+8 points to the data in array 1112, and the index i+16points to the data in array 1113.

Calculator 1115 derives height data which is placed into Z-data array1121 and/or intensity values which are placed into intensity-data array1122 (in another embodiment such as shown in FIG. 4D, intensity data1123 is obtained from a 2D scan camera—for example the middle line 2 ofthe trilinear array 423 is scanned out after illumination strobe fromnon-patterned light source 403 (e.g., a ring light) that is interleavedwith a strobed projector 404 that provides patterned light for 3Dmeasurements).

Feature recognition block 1125 uses data from array 1121 and/or array1122 to identify features of devices 99 in the refined data (such asedges of parts) and performs the necessary transformations (such asmasking and rotations of an individual part's data) needed to convertthe data into a standard form 126. This standard form data 126 is thenpassed to comparators 1136 and/or 1137 where the data are compared tothe criteria for good data from z-data criteria array 1131 and/orintensity-data criteria array 1132. The outputs 1138 and/or 1139respectively of comparators 1136 and 1137 are then optionally used toselect good parts, discard bad parts, and/or provide feedback to adjusta manufacturing process to a desired state.

Since the present invention provides for acquiring height {z} data forevery {x,y} point in the scanned region (in contrast to other systemswhich scan a single line down a row of pins, for example) the presentinvention allows for random orientation of parts in trays, withoutrequiring registration, shaking of trays, or other operations to preparethe parts for inspection. Rather, parts may be in any orientation in thescanned field of view.

Further, the present invention provides for random location of features(rather than aligned parts in trays, use of pocket edges or therequirement of rows of features such as pins or balls). There is norequirement for regularity in the configuration of features to bescanned (i.e., rows of pins). Indeed, even objects such as eggs orgemstones may be measured with great accuracy and speed. The presentinvention does not require features to be lined up in order to measurethose features quickly. In contrast, other systems which, for exampleuse laser triangulation, benefit from arranging parts, such that all thepins to be inspected are aligned with the line that the laser scanningbeam is directed. The present invention handles irregular spacing offeatures such as are found on chip-on-module and hybrid packages, whichoften lack regular spacings of features such as pins in a row.

In some embodiments of the present invention, crossed pattern projectors(such as shown in FIG. 6 and FIG. 8) are used for shadow reduction,i.e., a second projector provides measurements for areas that are in ashadow relative to the first projector.

In some embodiments, position or velocity detection is provided. In onesuch embodiment, a high-resolution linear position encoder is used tospecify the times at which line scans are taken, (in one suchembodiment, fiduciary stripes are applied to the field being scanned,such as to the edge of the clamp holding the tray of devices 99). Forexample, the start of the electronic “shutter” for each line scan issynchronized to a position. The scan timings thus are directly relatedto position rather than velocity, and thus make the measurement accuracyvelocity-independent. In one such embodiment, however, the scanningvelocity is maintained at a constant velocity, in order that a fixedshutter duration obtains the same amount of light. In another suchembodiment, both the start and stop timings of the shutter aresynchronized to position (e.g., using a linear position encoder). In yetother embodiments, the shutter duration is adjusted by a servo circuitthat detects the instantaneous or short-term integrated light output ofthe light source 402, in order to achieve greater accuracy, as describedfurther below.

In some embodiments, the present invention provides detection ofsubstrate warpage as well as ball top coplanarity. Since all {x,y}points are measured for z height, a measurement of the substrateplanarity or warpage is provided by the same scan that provides anindication of ball-top coplanarity. In some such embodiments, twodifferent measurements are made in the same scan (such as describedabove, by varying the beam-split amount, or the projection strobeintensity, or the imager shutter time, the dim features, such assubstrate details and planarity can be measured in interleaved scanswith the high-brightness features such as shiny solder balls). In someembodiments, two discrete apertures (lamp brightnesses, shutterdurations, or flash lengths) interleaved (e.g., a long aperture for darkportions or features, and a short aperture for bright or shiny features;e.g., one brightness for ball tops, another for substrate warp) areused.

FIG. 12 is a schematic layout of another preferred embodiment of thevision system 1200. The function of the layout is to project a sinewavepattern onto an object 1201 and to measure the reflected intensitieswith a tri-linear CCD without color filters. The vision system 1200includes a Dalsa camera 1210, a flat mirror 1220, a telecentric lenspair 1230 and 1231, a projection lens 1240 a grating 1250, a condenserpair 1260, a filament 1270 and a spherical mirror 1280. The CCD used bythe Dalsa camera 1210 is the Kodak KLI-2103 which contains 3 rows of2098 active photosites. Each photo site measures 14 μm square and thecenter to center spacing between the rows is 112 μm or the equivalent of8 pixels.

The field of view (FOV) of the telecentric lens 1230, 1231 is 2.25″which is wide enough to inspect two 27 mm parts (including a separationgap of 0.125″). This translates into a maximum average magnification ofm=0.514. The minimum allowed average magnification is m=0.499 (3%decrease) and the allowed magnification variation along the central axis(for one lens) is +/−0.5%, but is preferred to be less than +/−0.2%(which is equivalent +/−½ LSB of the range measurement). Compensationcan be added to the range calculations to reduce the affect ofmagnification variation if the variation is greater than +/−0.2%. Thedegree of telecentricity (that is, how parallel is the central axis ofthe apertures across the FOV) must not change more than 0.01 degreesover an 8 mil position change in the object 1201 plane. The positiondistortion must not exceed +/−1% of the FOV along the central axis.Ideally the position distortion should be less than +/−0.1%, but thiscan be obtained by software compensation if the lens is unable toprovide it. The maximum aperture opening must be at least f5.6 andpreferably f4.0. The aperture should be adjustable.

The grating 1250 is a sinusoidal line pattern. The grating 1250 isavailable from Sine Patterns LLC. The line pattern is oriented parallelto the 3 rows of the CCD. The frequency of the sinusoid and themagnification of the projection lens is chosen so that one cycle alongthe vertical imaging axis is 25.6 mils long to give a range resolutionof 0.1 mils.

The projection lens 1240 magnification is chosen so that one cycle alongthe vertical imaging axis is 25.6 mils long. The maximum aperture mustbe at least f4.0 and possibly as large as f2.0. The aperture is notrequired to be adjustable. The magnification change across the centralaxis must be +/−0.5% or less and preferably less than +/−0.2%. The axisof the lens is rotated to provide an extended depth of focus of the linepattern in the image axis. The rotation is such that the grating, imageaxis and projection lens axis tri-sect per the above drawing.

The condenser lens pair 1260 collects light from the filament and imagesthe filament onto the aperture of the projection lens. The aperture sizeshould be at least f1.0.

The filament 1260 is L7420 from Gilway Technical Lamp. The filament 1260size is 11.8×4.6 mm and the power is 400 watts. Other filaments with asimilar power rating can be substituted.

The spherical mirror 1270 has a radius equal to its distance from thefilament. Its purpose is to reflect light to the condenser lens. Sincethe filament blocks the direct path, consideration can be given tocreating a virtual image of the filament adjacent to the real filament.

A reflecting IR filter (not shown in the above drawing) between thefilament and the condenser lens is required because the CCD has a poorMTF response in the IR range and to reduce spherical aberrations in theoptical path.

Focus adjustment must be provided so that the optimal focus of bothoptical paths occurs at the object.

Light Intensity Control

FIG. 13A is a schematic view of one embodiment of a light-intensitycontroller according to the present invention. Many of the elements ofthe projector 402 and the imaging system (or imager) 404 have beendescribed above. For the sake of saving space, the description of theprojector 402 and the imaging system 404 will not be repeated here.Rather, common numbers will be used to describe the elements needed. Themachine-vision system 401 is for inspecting the device 99 which movesrelative to the projector 402 and the imaging system 404. To obtain moreaccurate values when an image is acquired, the machine-vision systemincludes a light sensor assembly 1310 which receives light from thelight source 418. The light sensor assembly 1310, as shown in FIG. 13,includes a beam splitter 1312 which splits off a portion of the lightproduced by the light source 418. The beam splitter allows a portion ofthe light from the light source to pass and reflects another portion ofthe light to a light sensor 1314. The beam splitter 1312 is positionedbetween the light source 418 and the device under test 99. Although thebeam splitter 1312 could be positioned anywhere along the path betweenthe light source 418 and the device under test 99, as shown in FIG. 13A,in this embodiment, the beam splitter 1312 is positioned between thelight source 418′ and the light source imaging element 414. The beamsplitter 1312 also serves a second purpose in some embodiments:filtering out certain undesirable light such as infrared light or lightof other wavelengths. In one such embodiment, beam splitter 1312 servesto pass only a narrow band of light frequencies (i.e., substantiallymonochromatic light), in order to facilitate focussing.

The light sensor 1314 is positioned to receive light from the beamsplitter 1312. The light sensor 1314 is typically a photo diode whichproduces an output 1316 responsive to the intensity of the lightreceived at the light sensor assembly 1310. It should be noted that insome embodiments, light sensor 1314 is used without the beam splitter1312. In other words, in such embodiments, the light sensor is merelyplaced somewhere in the light from light source 418 to collect a portionof the light from the light source 418. In some such embodiments, thevalue of the output 1316 from the light sensor 1314 is higher if thelight sensor 1314 is merely placed in the light path.

In this embodiment, the output 1316 is a signal that is used as part ofa feedback control loop to control the intensity of the light, or theintensity of the light that will be received by imager 404. As shown inFIG. 13A, the output 1316 is input to an amplifier 1318 to produce anamplified output or control signal 1320. The machine-vision system alsoincludes a power supply 1330 and related a power supply controller 1332.The power supply controller 1332 is, in one embodiment, an independentcontroller associated with the power supply or is, in anotherembodiment, part of another controller. In other words, the tasks ofcontrolling the power supply can be assigned to another controller, suchas a computer 128, that controls other various aspects of themachine-vision system. The value of the control signal 1320 indicatesthe intensity of the measured light from the light source 418. Bycontrolling the power input to the light source 418, the intensity ofthe light produced by the light source is controlled. The power supplycontroller 1332 controls the amount of power delivered to the lightsource so that the light intensity is within a desired or selectedrange. The value of the control signal 1320 indicates the intensity ofthe light from the light source 418. When the value of the controlsignal 1320 falls below a specified value which is outside the selectedor desired range, the controller 1332 increases the power input 1334 tothe light source 418. This in turn increases the intensity of the lightand the signal output from the light sensor 1314 and the control signal1320. If the value of the control signal 1320 is above a specified valuewhich is outside the selected or desired range, the controller 1332decreases the power input 1334 to the light source 418. This in turndecreases the intensity of the light from the light source 418 anddecreases both the signal output from the light sensor 1314 and thecontrol signal 1320. In other such embodiments, control signal 1320 isused to control the length of time that light source 418 is “on,” thuscontrolling the duration and/or shape of the light pulse output.

It should be noted that the are a number of ways contemplated by thepresent invention to control the intensity of the light from the lightsource 418 as received by imager 404. FIG. 13B shows a second preferredembodiment of this invention. The arrangement of most of the elements ofFIG. 13B are the same as the arrangement of the elements in 13A. Thedifference is that the control signal 1320 from the light sensor 1314and the amplifier 1318 is not used to control the light output fromlight source 1312, but rather in this second preferred embodiment, thecontrol signal 1320 from the light sensor 1314 is used to control thelength of time for image acquisition at the image detector 422, called“shuttering.” In one embodiment, the image detector 422 includes threerows of pixels also known as a trilinear array 423. In the preferredembodiment, the trilinear array 423 is comprised of rows ofsemiconductor pixels or photo diodes which are part of a charge-coupleddevice, such as a high-speed digital CCD camera available as a KODAK KLI2130 trilinear array.

The control signal 1320 from the light sensor 1314 is routed through atiming controller 1344 associated with the trilinear array 423 of theimage detector 422. The trilinear array 423 is a charge-coupled device.In this particular charge-coupled device, photodiodes are the pixels. Asthe photodiodes are exposed to light, the charge on the associatedcharge-coupled device builds until the timing controller 1344 removesthe charge at the end of an image acquisition. Thus, by controlling theamount of time the charge-coupled device is charged, the values of thecharge-coupled device or intensity of the light acquired at the imagedetector 422 can be controlled. The value of the control signal 1320indicates the intensity of the light from the light source 418. When thevalue of the control signal 1320 falls below a specified value which isoutside the selected or desired range, the timing controller 1344increases the length of the acquisition time at the image detector 422.This in turn increases the intensity of the light captured during theacquisition of the image at the image detector 422. If the value of thecontrol signal 1320 is above a specified value which is outside theselected or desired range, the timing controller 1344 decreases thelength of the acquisition time at the image detector 422. This in turndecreases the intensity of the light captured during the acquisition ofthe image at the image detector 422. As mentioned above the timingcontroller 1344 may be independent. More likely the timing controller1344 is part of another controller associated with the machine-visionsystem.

It should also be noted that the light, sensor assembly 1310 in otherembodiments is placed in the path of reflected light between thedevice-under-test 99 and the image detector 422. In addition, it shouldbe noted that the light sensor 1314 need not be linked with a beamsplitter 1312, as is shown in FIG. 13C. In FIG. 13C, the sensor 1314 isplaced outside the path of the elements of the projector 402. In otherwords, the light sensor 1314 does not interfere with the light that isprojected onto the device 99. The light source 418 sends light beams outin many radial directions. As a result, detecting the light outside thepath of the elements of the projector 402 is often equally as effectiveas sensing the light within the path of projected light using a beamsplitter (as shown in FIGS. 13A and 13B). Once sensed, the signal can beused to vary the power (similar to the embodiment shown in FIG. 13A) orcan be used to vary the timing during acquisition of the image at thetrilinear array 423 (similar to the embodiment shown in FIG. 13B). Forthe sake of illustration, a sensor 1314 of FIG. 13C is used to controlthe time of image acquisition.

In yet other embodiments, a sensor 1314 is used to control an aperturesize, a shutter-open time, focus divergence, or a light-transmissiondensity (or percentage) of an element (such as a liquid-crystal element)located in the light path between light source 418 and imaging element422. Thus, in various embodiments, the control signal from sensor 1314is used to control the light intensity of light source 418 (whethercontinuous or pulsed), the pulse length of light output by light source418 (if pulsed), the light passed along the light path (whether variedby aperture, density, length-of-light-path (divergence) and/or shuttertime), and/or the length of time for image acquisition of imagingelement 422. In still other embodiments as described below, the controlsignal from sensor 1314 is used to control a calculated correction ofthe output signal from imaging element 422 (in one such embodiment, ananalog adjustment is made to the analog output signal of imaging element422; in another such embodiment, a digital calculation adjustment ismade to the signal after it has been converted into a digital value).

Calculated Corrections

FIG. 14A is a schematic view of the imaging system 400 having a memorydevice 1400 associated with the trilinear array 423 of the imagedetector 422. As mentioned above, the photo diodes of the trilineararray 423 accumulate or discharge electrical charge to associatedtransistors during the time of acquisition. When the timing controller1344 sends a timing signal, the electrical charge values associated withthe transistors associated with each pixel are moved into the memorydevice 1400. Each of the pixels may have slightly different values inthe presence of the same amount of light. There can be many reasons forthis, including current bleeding from a nearby pixel or manufacturingtolerances associated with making the charge-coupled device. In oneembodiment, the charge-coupled device associated with the image detector422 is calibrated by exposing the pixels to an equal intensity light fora selected amount of time. The different values obtained are then storedin the memory device 1400.

From this stored information, a look-up table 1410, as shown in FIG.14B, is constructed that contains correction values 1412 for each of thepixels and associated transistors of the trilinear array 422. Severalcalibration tests may be conducted to assure that the correction values1412 are correct. The correction values may be multipliers used tomultiply a particular value 1412 or may be a value 1412 which is addedor subtracted from the value to obtain a proper value. Some values maycontain both a multiplication component and an additive component. Thetable lookup 1410 is housed within memory and used to apply correctionvalues to the values 1412 associated with the pixels of a trilineararray 423. In operation, the data from the pixels is acquired and placedin the memory device 1400. After acquisition, the correction value forthe particular pixel is applied to the data in memory to calculate thecorrected value for each pixel. These corrected values can be used andportray a more accurate image. One or more correction values can bestored in the table lookup 1410 for a particular pixel. In someinstances, where the light source 418 is strobed with more than oneintensities, there may be correction values associated with each lightintensity for the particular light. Furthermore, if a different light isused, such as a ring light, there could also be still differentcorrection values associated with each of these lights and the conditionunder which the lights are used. In summary, a table lookup can beconstructed in which there are correction values for various lights usedunder various conditions.

Now turning to 14C, another way to correct for variations in intensityof the light can be accomplished using the arrangement shown. After thecorrected values for each pixel in a trilinear array have beencalculated, the control signal 1320 from a light sensor 1314 can be usedto apply an overall correction value to the corrected values in memory1410. Of course, the overall correction value could also be applied tothe actual readings in memory 1410 initially and then the correctionvalues from the table lookup could be applied. This would yield the sameor an equivalent value to the method discussed previously in thisparagraph.

Thermoelectric Cooling of the Image Detector

FIG. 15 is a schematic view of the trilinear array 423 with athermoelectric cooling element 1500 associated therewith. The thermalelectric cooling element 1500 is attached to the trilinear array 423using a thermal conductive adhesive. The thermal electric coolingelement 1500 includes a cool side 1510 and a hot side 1520. The coolside of the thermal electric element 1500 is attached to the trilineararray 423. The thermal electric cooling element 1500 also includes atemperature sensor 1530. The temperature sensor 1530 outputs a signal1532 which is amplified by amplifier 1534 to produce a control signal1536. Control signal 1536 is fed back to a supply of power 1540. A powercontroller 1542 within the power supply 1540 decreases or increases thepower supplied to the thermal electric cooling element 1500. Byincreasing the power input to the thermal electric cooling element 1500,the temperature along the cool side can be lowered, thereby lowering thetemperature of the trilinear array 423. If too low a temperature isdetected by temperature sensor 1530, the control signal 1536 willreflect that the amount of power input to the thermal electric coolingelement 1500 must be lowered. The power controller 1542 can be withinthe power supply 1540 or can be part of another controller associatedwith the system.

It should be noted that the hot side of the thermal electric coolingelement 1500 may be provided with heat fans or other elements used toincrease convective or radiative cooling along the hot side 1520 of thethermal electric cooling element 1500. The basic operating principle ofthe thermal electric cooling element 1500 is the absorption orgeneration of heat as a current passes through a junction of twodissimilar metal materials. Electrons passing across the junction absorbor give up an amount of energy equal to the transport energy and theenergy difference between the dissimilar materials' conduction bands.Cryogenic temperatures are reached using heat rejected from one thermalelectric cooler stage to supply thermal input to the stage below. Thebasic operating principle of the thermal electric cooler is known as thePeltier Cooling Effect.

Advantageously, using thermoelectric cooling prevents the trilineararray 423 from heating up as they operate. As a result, the trilineararray 432 does not heat and the signals produced by the array 423 do notshift or vary, or only shift or vary slightly. This eases the task ofcorrelating the data obtained and also prevents the current associatedwith a dark area, called dark currents, from rising into a range wherethe current becomes noise. This elimination or prevention of noise alsosimplifies processing of the image.

Strobing Lamps to Stop Motion Smear

Returning to FIG. 13A, strobing the light source 418 and strobing thering light 403 will now be discussed. Strobing the light source 418 andstrobing the ring light 403 substantially reduces or stops motion smear.Motion smear is similar to a blur in a photograph. In order to get aproper amount of light for charging the trilinear array 423 to a levelwhere it produces useful data, a certain amount of light is needed.Without a strobe or high intensity, short burst of light, light must begathered over a selected amount of time. As mentioned previously, thetrilinear array 423 moves with respect to the devices 99 such that overthe selected amount time so that enough light is provided to thetrilinear array 423 for processing into useful data. When there is noburst of high intensity light, the device moves as the light is gatheredfrom the device 99. The end result is that the data, when processed,produces a blurred image since the device 99 has moved appreciably overthe time the necessary amount of light was produced.

To eliminate this blurring, which is also called motion smear, the lightsource 418 and the ring light 403 are “strobed.” In other words, acircuit is used which produces a short burst of high intensity light.This shortens the amount of time over which the device 99 can move withrespect to the light receiver or trilinear array 423 during imageacquisition. This lessens or eliminates motion smear in the acquiredimage. In one embodiment, LED light sources are used in order to obtaina precise and repeatable amount of light, and the circuit used to strobeor pulse the light source is shown in FIG. 10 of U.S. Pat. No.5,745,176. U.S. Pat. No. 5,745,176, in its entirety, is incorporatedherein by reference.

Mechanical Aspects of the Machine-Vision Station

FIG. 16 is a perspective view of the machine-vision system 1600. Themachine-vision system 1600 includes a first inspection station 1610, asecond inspection station 1612, a third inspection station 1614, afourth inspection station 1616, a first pick-and-place station 1620, anda second pick-and-place station 1622. In this embodiment, pick-and-placeoperations are performed at two stations; in other embodiments,pick-and-place operations are not performed, or are performed at onlyone station, or are performed at more than two stations. Located betweeninspection station 1612 and inspection station 1614, is a tray inverter1900. The tray inverter 1900 may also be called the flipper. A firstgantry 1630 is located above inspection station 1610 and inspectionstation 1612.

In the embodiment shown, the first gantry 1630 includes a firstinspection camera 1631 and a second inspection camera 1632. Eachinspection camera 1631, 1632 is capable of acquiring a 3D image or a 2Dimage of the parts being inspected at the inspection stations 1610 and1612. In various embodiments, cameras 1631, 1632 (or any of theinspection cameras at other inspection stations described herein) areimplemented as one or more of the cameras described for FIGS. 3-13 above(i.e., each “camera” could be one 3D camera such as shown in FIG. 4A, ora 2D and a 3D camera such as shown in FIG. 13A). In yet otherembodiments, each camera 1631 and camera 1632 is mounted on a separate,independently operable gantry in order to be separately movable to thelocations and at the speeds best suited for each inspection operation.In some embodiments, each camera is implemented as a plurality ofcameras operated in parallel to achieve even higher inspection speeds.For example, in one embodiment, camera 1631 is implemented as four heads401 (as shown and described in FIG. 4A) mounted in a two-by-two squarearray, such that each head 401 need only acquire one fourth of the areato be scanned in a given period of time. In another embodiment, a 2Dcamera and a 3D camera are combined at a single inspection station (suchas, for example, that shown in FIG. 4D), in order that information fromthe 2D inspection can be used to assist in the automaticcomputer-recognition of features to be analyzed in the 3D inspection.

In the embodiment shown in FIG. 16, the inspection cameras 1631, 1632are mounted on a gantry arm 1634. The gantry arm 1634 is cantileveredover the inspection stations 1610 and 1612. The gantry arm 1634 includesa translation mechanism (not shown) which allows the first inspectioncamera 1631 to move along the length of the gantry arm 1634 and alsoallows the second inspection camera 1632 to move along the length of thegantry arm 1634, both under the control of a control computer such ascomputer 128 of FIG. 1 or computer 1010 of FIG. 10. In the preferredembodiment, the translation mechanism allows the first camera 1631 tomove independently of the second camera 1632. In other words, the firstinspection camera 1631 can move along the gantry arm 1634 at a firstspeed while the second inspection camera 1632 can translate along thegantry arm 1634 at a second speed. The inspection cameras 1631 and 1632could also move in different directions along the gantry arm 1634. It iscontemplated that one inspection camera 1631 could do a 3D-typeinspection while the other inspection camera 1632 could do a 2Dinspection. Generally, the cameras 1631 and 1632 move at differentspeeds when doing a 2D inspection or 3D inspection. Therefore, it iscontemplated that while one of the cameras 1632 or 1631 is working in a2D mode, the other of the cameras 1631 or 1632 could be simultaneouslyworking in a 3D mode.

In one embodiment, the first inspection camera 1631 and the secondinspection camera 1632 are spaced such that both inspection cameras 1631and 1632 are positionable over a single station 1610 or 1612. As aresult, in this embodiment, it is necessary to move the gantry 1632 froma position over the first inspection station to a position over thesecond inspection station 1612. A driving mechanism allows such movementand also allows for various positions for inspection where one of thecameras 1631 may be positioned over a first inspection station 1610 andthe other of the cameras 1632 may be positioned over a second inspectionstation 1612.

In one embodiment, the devices being inspected with this particularmachine-vision system 1600 are semiconductor devices and, moreparticularly, semiconductor devices known as ball grid arrays (in otherembodiments, leaded devices such as quad-flat-packs and/or DIP(dual-in-line) packages are inspected). Ball grid arrays are becomingincreasingly popular semiconductor packages since the input/output pinsare short and capable of being densely packed on a semiconductorpackage. In addition, the ball grid array is a rugged semiconductorpackage. Each device has a series of balls or solder balls positioned onone side of the semiconductor package. It is very important that theseballs are uniform in shape and height. As a result, manufacturers go togreat lengths to achieve this result. A number of individualsemiconductor packages are carried in an industry-standard tray. Onestandard-formulating body is known as JEDEC and therefore one such typeof industry-standard trays are known as JEDEC trays. When “JEDEC trays”are discussed in the embodiments below, it is to be understood thatother embodiments are suitable for other types of trays and that othercontainers are used to hold the devices being inspected in variousembodiments of the present invention. In still other embodiments,devices attached to a carrier strip (such as plastic film) areinspected. In yet other embodiments, the devices being inspected are notplaced in any container, but are otherwise moved into place forinspection, for example, on a conveyor belt.

In the embodiment shown in FIG. 16, at the first inspection station1610, the JEDEC trays are presented to an inspection surface which ispositioned at a set distance from the inspection cameras 1631 and 1632on the gantry 1630 above the inspection station 1610. The semiconductordevices in the JEDEC tray are all positioned with the ball side of theball grid array devices presented for inspection (i.e., with electricalleads upwards, also called the “dead-bug orientation”). At the firstinspection station 1610, a 3D image of the balls of each of thesemiconductor devices in the JEDEC trays is obtained. The 3D inspectionis used to gather information with respect to the height of the balltops (e.g., the co-planarity of the top of all the balls), among otherthings (such as the shape and/or position of the balls, the position ofthe balls with respect to features on the substrate, etc.). At theinspection station 1612, a two-dimensional or 2D image of the ball sideof the semiconductor devices in the JEDEC trays is obtained. The 2Dinformation is useful in determining blob size, among other things.After the three-dimensional inspection at station 1610 and after thetwo-dimensional inspection at station 1612, the JEDEC trays are invertedso that the side of devices 99 opposite the balls can now be inspected(also called the “live-bug side” of the devices 99). The tray inverteror flipping mechanism 1900 will be discussed in further detail belowwhen FIG. 19A and FIG. 19B are described in further detail, below.

In one embodiment, at inspection station 1614, the top of the package isinspected with a 3D camera 1631′. Such package inspection includeschecking the dimensions of the package, whether there are chips, cracks,scratches or voids, among other things. In this embodiment, atinspection station 1616, the markings are inspected with a 2D camera1632′. In one embodiment, each of the ball-grid-array semiconductordevices in the JEDEC tray is marked with a model number and serialnumber as well as the manufacturer's identification so that the partsmay be tracked.

In the embodiment shown, a second gantry 1630′ is positioned over theinspection station 1614 and inspection station 1616. The gantry 1630′includes a gantry arm 1634′. A first inspection camera 1631′ and asecond inspection camera 1632′ are mounted to the gantry arm so thateach of the inspection cameras 1631′ and 1632′ can move independently ofthe other inspection camera. The gantry 1634′ can also move betweeninspection stations 1614 and 1616 so that the various inspections may beaccomplished at those stations. In one embodiment, the second gantry1630′ and inspection camera 1631′ and 1632′ are essentially the same asthe first gantry 1630.

In other embodiments, such as described for FIG. 10 above, othercombinations of one or more 2D and/or 3D cameras are provided for theinspection of the front, backs, and/or sides of devices 99 beinginspected. In such embodiments, one or more inspection stations,flipping stations, and/or sorting (pick-and-place) stations areprovided.

In this embodiment shown in FIG. 16, the trays of devices are movedquickly into position, and then stopped, in order that the parts do notmove in the trays. Thus, the parts remain stationary, and the camera(s)(e.g., camera 1631, 1632, 1631′ and 1632′) are moved to perform thescanning operation.

In yet other embodiments, the devices are moved and the scanning cameraremains stationary. In one such embodiment, the devices 99 are mountedto a carrier strip (e.g., a suitable plastic film), and the carrierstrip and devices are moved in a series in the optical path of one ormore 2D and/or 3D scanning cameras (such as, for example camera head 401of FIG. 4A.

After the inspection of the various parts at inspection stations 1610,1612, 1614 and 1616 are completed, any bad parts or parts failinginspection are noted, as well as the position of the parts within theparticular tray. The tray with good parts and bad parts is then movedfrom inspection station 1616 to pick-and-place station 1620. If there isnot a tray located at pick-and-place station 1622, the tray of good andbad parts will be moved to pick-and-place station 1622 and anotherinspected tray of good and bad parts will be moved to pick-and-placestation 1620. A vacuum pickup mechanism 2000 is used to remove bad partsfrom the trays located at pick-and-place station 1620 and place theminto trays located at pick-and-place station 1622. The vacuum pickup2000 also removes good semiconductor devices from the trays located at1622 and places them into trays located at station 1620. In other words,the bad parts from trays at station 1620 are replaced with good partsfrom trays located at station 1622. Once all of the bad parts have beenreplaced in the tray located at station 1620, the tray of good parts isremoved (e.g., automatically under the control of computer 1010, seeFIG. 10 above) from that station and a new tray is positioned intostation 1620. In one embodiment, a compartment and elevator is locatedbelow the pick-and-place station 1620, as well as below thepick-and-place station 1622.

When all of the parts are bad in the tray 1622, the parts are thenremoved (e.g., automatically under the control of computer 1010, seeFIG. 10 above) from that pick-and-place station and into a compartment(e.g., lowered on an elevator) below that pick-and-place station.Therefore, all of the trays in the elevator and compartment belowpick-and-place station 1622 will be trays of bad parts, while all of thetrays below pick-and-place station 1620 will be trays of good parts thathave successfully passed the inspection at the previous inspectionstations 1610, 1612, 1614 and 1616. The pick-and-place mechanism 2000will be described in further detail with respect to FIG. 20. It shouldbe noted, however, that the pick-and-place station includes a vacuumpickup which uses a vacuum on the marked or label side of thesemiconductor devices in the tray to move them from tray to tray.

Below each inspection station 1610, 1612, 1614 and 1616, as well asbelow each of the pick-and-place stations 1620 and 1622, is acompartment and elevator mechanism 1710, 1712, 1714, 1716, 1720 and1722.

Now turning to FIGS. 17A and 17B, the elevator and compartmentmechanisms 1710, 1712, 1714, 1716, 1720 and 1722 of one embodiment willnow be discussed. FIG. 17A is a top view of the compartment and elevatormechanisms 1710, 1712, 1714, 1716, 1720 and 1722 of the machine-visionsystem 1600. FIG. 17B is a side view of one of the compartment andelevator mechanisms located below an inspection station. Since each ofthe compartment and elevator mechanisms 1710, 1712, 1714, 1716, 1720 and1722 are substantially the same, only the compartment and elevatormechanism 1712 will be described in detail. It is to be understood thatthe others have substantially the same parts or elements.

The compartment and elevator mechanism 1712 includes a compartment 1730.Within the compartment 1730 is an elevator mechanism 1740. The elevatormechanism includes an elevator floor, an elevator guide 1744, a motor1746 and a lifting mechanism 1748. The elevator motor 1746 is connectedto the lifting mechanism 1748 with a belt 1750. Thus, by turning themotor 1746, the lifting mechanism 1748 is then used to raise theelevator floor up or down within the compartment 1730. The elevatorguide 1744 and the lifting mechanism 1748 are positioned on the side ofthe elevator floor or elevator plate 1742. With the lifting mechanism1748 and the elevator guide 1744 positioned on the side of the elevatorfloor or elevator plate, the JEDEC trays which hold the semiconductordevices can be moved into the compartment onto the elevator floor andthen abutted to the back side of the compartment 1730 withoutinterference from the elevator guide 1744 or the lifting mechanism 1748.

Access to the compartment 1730 of the compartment and elevator mechanism1712 is gained through a door 1760. The door has a handle 1762. The door1760 pivots on a pivot point 1763. Also associated with the door 1760 isa solid door stop 1764 which stops the door in a substantially parallelposition with respect to the elevator floor 1742 when the compartment1730 is devoid of JEDEC trays. The door 1760 is shown in a closedposition, as well as an open position, in FIG. 17B. The compartment 1730is also devoid of JEDEC trays for the sake of illustration.

When the compartment 1730 is empty, the elevator floor 1742 is moved toa position where it is essentially or substantially level with respectto the door. The door 1760 includes a first tray guide 1770 and a secondtray guide 1772. The first tray guide 1770 includes a lip 1771. Thesecond tray guide 1772 also includes a lip 1773. When the door 1760 isopen, the surface of the first tray guide 1770 and the second tray guide1772 is almost or substantially level with the position of the elevatorfloor 1742 within the compartment 1730. The lips 1771 and 1773 of thedoor 1760 are dimensioned so that the JEDEC trays fit between the lips1771 and 1773 with a slight amount of clearance.

Advantageously, when an operator loads trays into the compartment 1730,the door 1760 can be opened and a stack of JEDEC trays can be placed onthe door and specifically on the first tray guide 1770 and the secondtray guide 1772. The bottom tray of the stack of JEDEC trays will fitbetween the lip 1771 associated with the first tray guide and the lip1773 of the second tray guide 1772. The operator can then merely slidethe stack of JEDEC trays into the compartment and onto the elevatorfloor 1742. By sliding them in all the way, the stack of JEDEC trayswill be placed against the back wall of the compartment 1730. Inaddition, the lips 1771 and 1773 on the door also are used to positionthe trays carrying the semiconductor parts within the compartment 1730.In other words, the lips 1771 and 1773 on the door 1760 position theJEDEC trays properly along the width dimension of the compartment 1730.When placed in the compartment using the door, the trays are very closeto the position that they will need to be in for presentation to thetray-transfer device 1800 above the compartment 1730, and at either aninspection station 1610, 1612, 1614 or 1617, or at a pick-and-placestation 1620 or 1622. The details of the tray-transfer device 1800 willbe discussed in further detail with respect to FIGS. 18A and 18B.

The elevator mechanism 1740 also includes a sensor 1780 or a series ofsensors 1780. The sensors 1780 determine the JEDEC tray as well as thenumber of JEDEC trays loaded with the elevator mechanism 1712. Signalsfrom the sensors are then used by a separate controller to control theelectric motor so that the top JEDEC tray can be moved to an inspectionstation, such as 1612 which is above the compartment and elevatormechanism 1712.

Now turning briefly to look at inspection station 1610 and thecompartment and elevator mechanism 1710 situated below the firstinspection station 1610, a special advantage of this particularinspection vision machine 1600 will now be discussed. The JEDEC trayswhich hold the devices to be inspected can be moved directly from thecompartment and elevator mechanism 1710 to the first inspection station1610. In fact, the elevator mechanism 1710 moves the top JEDEC tray tothe inspection surface associated with the inspection station 1610. Inorder to do this, the compartment and elevator mechanism 1710 must besubstantially aligned with the inspection station 1610.

This is advantageous because it reduces the footprint of themachine-vision system 1600, and thus the space needed on a factoryfloor. In previous systems, a separate station would be positionedbefore the first inspection station 1610 and the JEDEC trays areessentially lowered to the station so that it can be loaded into a firstinspection station such as 1610. Thus, previous systems required aseparate loading station to the side of the first actual inspectingstation. By using an elevator mechanism and compartment 1710 below thefirst inspection station 1610, there is no need for a separate loadingstation. An additional advantage is that all of the JEDEC trays arepositioned on a solid elevator floor 1742. In previous inspectionstations where there is a loading station, a set of pins is used to holdthe bottom tray which is lowered to a loading station. The disadvantagewith this previous system is that the pins, typically four or five,carry the entire load of all the JEDEC trays in the stack. Thus, thepins were much more prone to failure. In addition, the JEDEC trays werenot held as solidly and were stressed which, in turn, stresses the partslocated within the JEDEC tray. Using the solid elevator floor 1742provides for a much more reliable system, as well as a system that willnot stress the parts or the trays. In addition, the use of an elevatorraising the JEDEC trays up from below the inspection surface 1610,eliminates the need for a separate loading station.

A first tray-transfer device 1800, a second tray-transfer device 1802and a third tray-transfer device 1804 are used to move the JEDEC traysbetween the inspection station 1610, the inspection station 1612, theinspection station 1614, the inspection station 1616, the pick-and-placestation 1620, the pick-and-place station 1622, and the tray inverter orflipping mechanism 1900.

Now turning to FIGS. 18A and 18B, the tray-transfer devices 1800, 1802,1804 will now be further detailed. FIG. 18A is a top view of theinspection stations and the tray-transfer devices for moving traysbetween the various inspection stations, the pick-and-place stations,and the tray inverter or flipper mechanism 1900. FIG. 18B is a frontview of one of the tray-transfer devices. The gantries, as well as thepick-and-place mechanism 2000, have been removed from the top view ofFIG. 18A so that the tray-transfer devices can be more clearly shown.

As currently positioned, the tray-transfer device 1800 has a first JEDECtray 1810 positioned at inspection station 1610 and a second JEDEC tray1812 positioned at inspection station 1612. The first tray-transferdevice 1800 moves JEDEC trays between the first inspection station 1610,the second inspection station 1612 and the flipping mechanism 1900. Thesecond tray-transfer device 1802 moves JEDEC trays between the flippingmechanism 1900, the inspection station 1614 and the inspection station1616. The second tray-transfer device is holding JEDEC tray 1814 atinspection station 1614 and JEDEC tray 1816 at inspection station 1616.The third tray-transfer device 1804 is shown holding a JEDEC tray 1820at pick-and-place station 1620 and holding a JEDEC tray 1822 atpick-and-place station 1622. The tray-transfer device 1804 moves JEDECtrays between inspection station 1616 and pick-and-place station 1620and pick-and-place station 1622. Each of the tray-transfer devices 1800,1802, 1804 are essentially the same. Therefore, only one will bedescribed herein.

The tray-transfer device 1800 includes a plate 1830 which has a firstopening 1831 and a second opening 1832 therein. The openings 1831 and1832 in the plate 1830 are dimensioned so that a standard JEDEC tray,such as 1810 or 1812, will fit within the opening 1831 or 1832.Positioned around each of the openings is a series of engagement pinmechanisms 1840, 1841, 1842, 1843 and 1844. As shown in FIG. 18A, eachtray is engaged by five pin mechanisms 1840, 1841, 1842, 1843 and 1844.The pin mechanisms 1840, 1841, 1842, 1843 and 1844 engage standardopenings in the JEDEC tray 1812. It should be noted that the JEDEC traysare standard, so JEDEC tray 1810 or JEDEC tray 1812 will each be engagedin the same fashion.

Five pins are positioned around each opening 1831 and 1832. The firstpin to engage is pin 1842. The pin 1842 not only engages a portion ofthe JEDEC tray 1812, but also pushes the JEDEC tray toward or to a datumsurface 1834, associated with the opening 1832. The next two pins thatengage are along one edge of the JEDEC tray. In this particularinstance, the pins 1844 and 1843 engage the JEDEC tray 1812 next andpush the tray toward another datum surface 1836, associated with theopening 1832. It should be noted that this sequence could be reversedand that the most important aspect is that the JEDEC tray 1812 be pushedup against datum surfaces 1834 and 1836, as the pins 1842, 1843 engagethe JEDEC tray.

The final set of pins to engage the JEDEC tray are 1840 and 1841. Bypushing the JEDEC tray to the datum surface 1832 and to the second datumsurface 1836, the exact location of the tray is known. With the JEDECtray positioned against the datum surfaces 1834 and 1836 and with theplate 1830 also positioned against a datum surface, the exact locationof the JEDEC tray within the opening 1832, the exact location in spacewith respect to the gantry is known. A 2D or 3D inspection, or anyinspection for that matter, needs to have this data before theinspection can begin. Each of the pin mechanisms 1840, 1841, 1842, 1843and 1844 is capable of a first and a second position with respect to thesurface of the plate 1830. In the industry there are two standard JEDECtrays for holding electronic parts such as semiconductor ball grid arraydevices. A first standard JEDEC tray has a depth which is smaller thanthe second standard JEDEC tray. By having the pins 1840, 1841, 1842,1843 and 1844 positionable to one of two positions, the two standardwidth or depth JEDEC trays that the machine-vision system 1600 will berequired to handle, can be accommodated.

Also associated with the tray-transfer device 1800 is a tray drivermechanism 1860. The tray driver mechanism 1860 is driven by an electricmotor 1862. The tray driver moves flat plate 1830 of the tray-transferdevice 1800 to various positions between the inspection station 1610,the inspection station 1612, and the tray inverter or flipper mechanism1900. The plate of the tray transfer mechanism includes a series ofprecision bearings on the bottom surface. The plates essentially rollalong a precision surface associated with the machine-vision system1600. This allows the tray-transfer device 1800 to move between thestations 1610 and 1612 and the flipper or tray inverter mechanism 1900.A belt 1864 connects the driver mechanism 1860 to the electric motor1862.

A cable harness 1870 is used to provide the electronic control signalused to control the driver.

The tray inverter mechanism 1900 is used to turn the semiconductordevices housed or held by a JEDEC tray from one position to anotherposition. Essentially, the tray inverter moves the semiconductordevices, such as a ball grid array device, from the ball side to theflat side opposite the ball side, being exposed within the JEDEC tray.

Now turning to FIGS. 19A and 19B, the tray inverter mechanism 1900 willbe further detailed. FIG. 19A is a side view of the tray invertermechanism or flipper 1900. FIG. 19B is a front view of the tray invertermechanism or flipper 1900. The tray inverter mechanism 1900 includes afirst jaw 1910 and a second jaw 1912. The jaw 1910 and the jaw 1912 areessentially a flat plate. The jaws 1910 and 1912 could also each be aset of spaced-apart, cantilevered ends. The jaws 1910 and 1912 havethread engagement ends 1911 and 1913, respectively. The engagement ends1911 and 1913 are openings within the jaws 1910 and 1912, respectively,which are threaded so that they can ride on a threaded shaft 1920.

The threaded shaft includes a right-hand threaded portion 1922 and aleft-hand threaded portion 1924. The threaded shaft 1920 is turned by anelectric motor 1926. When the motor 1926 turns the threaded shaft 1920one way, the jaws 1910 and 1912 will move toward one another. When theelectric motor 1926 rotates the threaded shaft 1920 in the otherdirection, the jaws 1910 and 1912 will move away from each other.

Each of the jaws includes a set of engagement pins 1941, 1943 and 1944.Although only three engagement pins are shown for each jaw 1910 and1912, it should be understood that there is a fourth pin which is notshown. The four engagement pin mechanisms engage the side openings ofthe JEDEC tray. Associated with each of the jaws 1910 and 1912 is asensor 1957 and 1959. The sensors 1957 and 1959 sense the distancebetween the first jaw 1910 and the second jaw 1912. The sensors are usedto prevent a spill of all of the semiconductor devices carried within aparticular JEDEC tray that are about to be inverted.

The signals from the sensors 1957 and 1959 indicate how close they arewith respect to one another. If the two sensors 1957 and 1959 are notclose enough, the inverter will not try to attempt to invert the tray.If the sensors 1957 and 1959 are not close enough, the operator mustcheck to see if all of the semiconductor devices are within the JEDECtray.

One of the common things that happens when handling semiconductordevices within a JEDEC tray is that one or more of the semiconductorsmay become jostled and come out of the tray slightly and become pinchedbetween a tray held by the jaw 1910 and the JEDEC tray held by the jaw1912. In that event, the JEDEC trays would not be engaged with oneanother and inverting them would merely spill the remainingsemiconductor devices, causing a stoppage of the inspection system whilethe semiconductor devices are cleaned up. The sensors 1957 and 1959prevent this from occurring.

Also associated with the inverter device is a rotator 1960. The rotatormoves the jaws 1910 and 1912 substantially 180 degrees from one another.In other words, the rotator flips the first jaw from a “lower” positionto an “upper” position and flips another jaw from an “upper” position toa “lower” position. The rotator 1960 includes stops so that the trayswill essentially be flipped through 180 degrees. A rotator motor 1962 isused to drive the rotator. The rotator motor can also have stops or be amotor that works between 0 degrees and 180 degrees.

In operation, an empty JEDEC tray is held within the pin engagementmechanisms associated with the upper jaw 1912. A populated or full trayis moved from inspection station 1612 to a position over the lower jaw1910, as shown in FIGS. 19A and 19B, by the tray-transfer device 1800.The threaded shaft 1920 is turned so that the tray-transfer device canmove over the jaw 1910 without interference with pin-locking mechanisms1941, 1943 and 1944, as well as the pin mechanism which is not shown.The threaded shaft is then turned so that the jaw 1910 is travelingtoward the jaw 1912 until the pins 1941, 1943 and 1944 and the unshownpin are able to engage the populated JEDEC tray. Once the populatedJEDEC tray is removed from the tray-transfer device opening 1832, thethreaded shaft 1920 is turned in the opposite way to lower the tray fromthe opening 1832 in the tray-transfer device 1800. The tray-transferdevice or the plate 1830 of the tray-transfer device is then removedfrom between the jaws 1910 and 1912 of the tray inverter or flippingmechanism 1900.

After the tray-transfer device 1800 is clear of the tray inverter 1900,the threaded shaft is rotated so that the populated tray held by jaw1910 approaches and engages the unpopulated tray held by jaw 1912. Thesensors 1957 and 1959 assure that the two JEDEC trays properly engageone another so that electronic devices held within the two JEDEC trayswill not spill during the inversion process. Once the sensors 1957 and1959 indicate that the two JEDEC trays are properly engaged, the rotator1960 and the rotator motor 1962 are used to flip the JEDEC trays 180degrees. Once inverted, the now populated tray is removed from the jaw1912 using the tray-transfer device 1802.

The threaded shaft is used to lower the now populated JEDEC tray havingflipped-over devices to a point where the second tray-transfer device1802 can be moved over the jaw with the JEDEC tray thereon. The threadedshaft then moves the JEDEC tray up and into engagement with the opening1831, associated with the second tray-transfer device 1802. A pinmechanism also engages the side of the tray. It should be noted that thepins of the jaws are spaced to grab a different set of openings in theJEDEC tray than the pins of the tray-transfer devices. The tray-transferdevice is then used to move the tray to the inspection station 1814where the dimensions of the semiconductor devices are checked. Thepreviously populated JEDEC tray now becomes the new empty tray which isin the upper jaw. The process is repeated over and over as the JEDECtrays move down the line.

It should be noted that the JEDEC trays move in a direction which isparallel to the short dimension of the JEDEC trays. This, too, is anadditional advantage is keeping the footprint of the machine-visionsystem 1600 at a minimum. In other words, by transferring the traysalong a direction parallel to the short direction of the JEDEC trays,the linear dimension of the machine-vision system 1600 is shortened.

Now turning to FIGS. 19C, 19D, 19E, 19F, and 19G, and FIGS. 19H, 19I,19J, 19K, and 19L, the tray inverter mechanism 1900′ will be furtherdetailed. FIGS. 19C-19G are end views of the tray inverter mechanism orflipper 1900′ in various stages of operation. FIGS. 19H-19L arecorresponding side views of the respective FIGS. 19C-19G. FIG. 19C showsa jaw 1910 having a tray 89 populated with a plurality of devices 99.Jaw 1930 of FIG. 19C replaces jaw 1912 of FIG. 19A, and is operated in asimilar manner, except as described in more detail below. In oneembodiment, jaw 1930 has an operative surface that is a thin stiff sheetof a suitable metal. Tray 89 is held in jaw 1910 as described above forFIG. 19A. Jaw 1930 is lowered (and/or jaw 1910 is raised) until it isnear or touching the upper surfaces of devices 99, as is shown in FIG.19D. FIG. 19H shows a side-view schematic diagram of the embodiment ofinvertor mechanism 1900′. In this embodiment, jaw motor 1926 rotatesthreaded shaft 1925 to raise and lower jaw 1910. Jaw motor 1927 rotatesthreaded shaft 1923 to lower and raise jaw 1930. Slider motor 1928rotates threaded shaft 1929 to move jaw 1930 laterally in order to slideit out from under devices 99, as further described below for FIG. 19G.Some of the details are omitted in FIGS. 19C-19L for clarity ofexplanation.

Referring to the end view FIG. 19D (and corresponding side view of FIG.19I), both jaws, now in engagement, are rotated (e.g., using rotatormotors 1961 and 1962, shown schematically) in order to invert all thedevices 99, resulting in the orientation shown in FIG. 19E (andcorresponding side view of FIG. 19J). Jaw 1910 is raised (and/or jaw1930 is raised) until the two jaws are sufficiently separated, thusleaving the devices 99 resting with the previously upper surfaces ofdevices 99 now downward on jaw 1930, and tray 89 held in jaw 1910 awayfrom the devices, resulting in the orientation shown in FIG. 19F (andcorresponding side view of FIG. 19K). Jaw 1910 is then inverted andmoved to a position below jaw 1930 resulting in the orientation shown inFIG. 19G (and corresponding side view of FIG. 19L). In one embodiment,jaw 1930 is then slid laterally using slider motor 1928 and screw 1929,with a holder or pusher 1931 holding the devices in position above tray89 as jaw 1930 is moved out of the way. As jaw 1930 is slid out of theway, the devices 99 drop into tray 89, with their orientation invertedas compared to FIG. 19C. The jaws each then return to their respectiveoriginal positions, using their respective motors 1926, 1927, 1928 1961and 1962, resulting in the orientation shown in FIG. 19C (andcorresponding side view of FIG. 19H), and the tray of inverted parts isremoved from jaw 1910, in a manner similar to that described above. Inthis embodiment (shown in FIGS. 19C-19L), the second tray (as used inFIGS. 19A and 19B) is not used, but rather a quite similar invertingoperation is used to invert all the devices and place them back into thesame tray as they started in. In some embodiments, tray 89 is a pocketedtray, with a separate pocket associated with and holding each device. Inother embodiments, a container or tray without pockets is used.

In yet another embodiment, a configuration otherwise substantially thesame as shown in FIGS. 19C-19L is used, except that no tray is used. Afirst conveyor moves a plurality of devices 99 onto lower jaw 1910(similar to FIG. 19C, but without tray 89). Jaw 1930 is lowered onto thedevices (similar to FIG. 19D, but without tray 89), and the two engagedjaws rotate to invert all of the devices in a single operation (similarto FIG. 19E, but without tray 89). A conveyor (either the firstconveyor, or a second conveyor) then removes the devices from jaw 1930.

Now turning to FIG. 20, the picker and placer 2000 will be furtherdetailed. FIG. 20 shows a pick and placer for replacing bad devices onJEDEC trays located at inspection station 1620 with good devices foundin JEDEC trays at station 1622. The pick-and-place robot includes an arm2010 and a vacuum pickup 2020. The vacuum pickup 2020 is placed on oneend of the arm 2010. The other end of the arm includes a first ballscrew device 2030 and a second ball screw device 2032. The first ballscrew device translates the arm 2010 and the attached vacuum pickup 2020along the length of the JEDEC trays at the pick-and-place stations 1620or 1622. The second ball screw device 2032 moves the arm 2010 and theattached vacuum pickup 2020 up and down with respect to the inspectionsurface and up and down with respect to the devices within the JEDECtrays. The pick-and-place mechanism can also be moved controllably alongthe short dimensions of the trays so that the pick-and-place device 2000can remove bad parts from the inspection station 1620 and place them inthe bad trays located at station 1622, and then pick up a good part fromstation 1622 and replace the previously bad part in the JEDEC tray atstation 1620.

Advantageously, the footprint of the machine-vision system forinspecting parts is smaller since an additional station is not neededfor loading trays to the first inspection station. In addition, thetrays are loaded onto a tray-transfer device so that the direction oftravel along the tray-transfer device is along the shorter dimension ofthe trays. This provides for a shorter line and a smaller footprint.Another advantage is that the inspection can take place automaticallywithout the intervention of a human. This lessens the chance foroperator error during the inspection process.

FIG. 21 shows an acquired image 2100 showing various heights of a ball2110 being inspected. The ball 2110 includes shadowed portions depictedby the black areas in the acquired image.

CONCLUSION

A machine-vision system 100 for inspecting a device is disclosed. Thedevice has a first side and a second side. The machine-vision system 100includes a first inspection station 1832 for inspecting a first side ofa device and a second inspection station 1814 for inspecting a secondside of a device. The machine-vision system 100 also includes atray-transfer device that operates to move the device from the firstinspection station 1832 to the second inspection station 1814. Thetray-transfer device also includes an inverting mechanism 1900 thatoperates to invert the device so that the first second side of thedevice can be inspected at the first inspection station 1832 and thesecond side of the device can be inspected at the second inspectionstation 1814. In one embodiment the inverting mechanism 1900 ispositioned between the first inspection position 1832 and the secondinspection position 1814. The inverting mechanism 1900 includes amechanism for flipping the devices 1900 carried in a tray. The mechanismfor flipping the devices 1900 includes a first jaw 1910 having a surfacefor receiving a first tray, and a second jaw 1912 having a surface forreceiving a second tray. The mechanism for flipping the devices 1900 hasa mover 1926 for moving the first jaw, the first tray having a pluralityof devices, the second tray, and the second jaw 1912 into engagementwith each other. The first tray is generally associated with the firstjaw 1910 and the second tray is generally associated with the secondjaw. The mechanism for flipping the devices 1900 also has a rotator1961, 1962 for rotating the first and second jaw. In the machine-visionthe mover 1926 moves the first jaw 1910 in a direction substantiallyperpendicular to the surface for receiving a tray associated with thefirst jaw 1910. In the machine-vision system 100 the mover 1926 alsomoves the second jaw 1912 in a direction substantially perpendicular tothe surface for receiving a tray associated with the first jaw. Theinverting mechanism 1900 moves the plurality of devices to the secondtray such that the second sides of a plurality of devices are presentedfor inspection. The rotator 1961, 1962 of the inverting mechanism 1900can also be said to move the plurality of devices to the second traysuch that the second sides of a plurality of devices are presented forinspection. The inverting mechanism 1900 is adapted to place theplurality of devices in the second tray at the second inspectionstation. The tray transfer device of the machine-vision system 100includes means for moving the tray with respect to the invertingmechanism 1900. The machine-vision system 100 may also include a picker2010 for picking devices which fail inspection from the second tray.

In another invention the machine-vision system 100 for inspecting aplurality of devices positioned within a plurality of device-carryingtrays may also include a first tray adapted to carry a plurality ofdevices, a second tray adapted to carry a plurality of devices and aflip station 1900 for flipping the plurality of devices carried in afirst tray from a first inspection position in the first tray to asecond inspection position in the second tray. The flip station 1900further includes a first jaw 1910 having a surface for receiving a firsttray, and a second jaw 1912 having a surface for receiving a tray. Amover moves the first jaw 1910, a first tray having a plurality ofdevices, a second tray, and the second jaw 1912 into engagement witheach other. The first tray is generally associated with the first jaw1910 and the second tray is generally associated with the second jaw. Arotator 1961, 1962 for rotates the first and second jaw. Themachine-vision system also includes a first slide clamp for holding atleast the first tray. The first slide clamp is for moving the first trayfrom a first inspection station 1832 to a flip station. Themachine-vision system also includes a second slide clamp for holding atleast the second tray. The second slide clamp moves the second tray fromthe flip station 1900 to the second inspection station. The flip station1900 of the machine-vision system 100 further includes a mechanism forflipping the devices carried in a tray. The mechanism includes means forlimiting the motion of the rotator. The mover moves the first jaw 1910in a direction substantially perpendicular to the surface for receivinga tray associated with the first jaw. The mover moves the first jaw 1910and the second jaw 1912 in a direction substantially perpendicular tothe surface for receiving a tray associated with the first jaw.

Another invention is directed toward a flipping mechanism fortransferring a plurality of devices from a position in a first tray to aposition in a second tray. The flipping mechanism includes a first jaw1910 having a surface adapted to receive the first tray, a second jaw1912 having a surface adapted to receive the second tray, a mover formoving the first jaw, the first tray, the second tray, and the secondjaw 1912 into engagement with each other, said first tray associatedwith the first jaw 1910 and the second tray associated with the secondjaw and a rotator 1961, 1962 for rotating the first and second jaw. Themover can be controlled to remove the first tray from a first inspectionsurface. The mover can be controlled to place the second tray at asecond inspection surface.

Yet another invention is directed to a method for acquiring physicalinformation associated with a plurality of devices placed in a tray. Themethod comprising the steps of inspecting a first side of a devicewithin a first tray, removing the first tray from a first surface andplacing the first tray at a flip station, moving a second tray to aposition near the first tray, flipping the first tray and second tray tomove the device from the first tray to the second tray and place thedevice in the second tray so that a second side of the device ispresented in the second tray, and inspecting a second side of the devicewithin the second tray. The method further includes the step of movingthe second tray to a second inspection surface.

Another invention is to a machine-vision system for inspecting aplurality of devices and for inverting the plurality of devices frombeing positioned in a first tray, the machine-vision system having afirst jaw 1910 with a surface for receiving the first tray, and a secondjaw 1912 having a surface, and a mover for moving the first jaw, thefirst tray having a plurality of devices, and the second jaw 1912 intoengagement with each other, said first tray associated with the firstjaw. The invention also includes a rotator 1961, 1962 for rotating thefirst and second jaw. The machine-vision system also includes a firstconveyer for moving the first tray having a plurality of devices thereinto the first jaw, and a second conveyer for moving the first tray havinga plurality of devices therein from the first jaw. The first jaw 1910 iscapable of holding, in any position, a tray devoid of devices. Themachine-vision system further includes a slider for transferring theinverted devices from the second jaw 1912 into the first tray.

Advantageously, the machine-vision-inspection system is capable offlipping trays of devices without operator intervention. The flippingdevice operates to reliably flip the trays of devices so that allportions of the devices within the trays are reliably inspected. Theflipping device that facilitates automated high-speed three-dimensionalinspection of objects in a manufacturing environment. The flippingmechanism is also easily accommodated as a station on an automatedmanufacturing line. Yet another advantage is that the inspection cantake place automatically without the intervention of a human. Thislessens the chance for operator error during the inspection process andaids in the throughput of the machine-vision-inspection system.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other embodiments will beapparent to those of skill in the art upon reviewing the abovedescription. The scope of the invention should, therefore, be determinedwith reference to the appended claims, along with the full scope ofequivalents to which such claims are entitled.

1. A machine-vision system for inspecting devices in trays, each traypopulated with a plurality of devices, the system comprising: a firstinput station having a first input elevator configured to hold aplurality of trays, wherein the plurality of trays includes a first trayheld at an inspection position of the first input elevator that is aboveall other ones of the plurality of trays held by the first inputelevator, and wherein each one of the plurality of trays has a firstplurality of that tray's devices that are yet to be inspected; a firstoutput station having a first output elevator for holding a plurality oftrays; and a first machine-vision imager that performs a firstmachine-vision inspection of the first plurality of devices in the firsttray held at the inspection position of the first input station. andwherein, upon completion of the first machine-vision inspection, thesystem operates to then move the first tray, which has the firstplurality of that tray's devices that have been inspected by the firstmachine-vision imager, off the inspection position of the first inputelevator and eventually to the first output elevator, and tosuccessively raise the other ones of the plurality of trays to theinspection position of the first input elevator to be inspected by thefirst machine-vision imager.
 2. The system of claim 1, wherein thesystem further comprises: a second machine-vision inspection stationthat inspects a second side of each of the first plurality of devices,wherein the first inspection station inspects a first side of each ofthe plurality of devices in the first tray, and wherein the system movesthe first tray to the second inspection station after the completion ofthe inspection at the first inspection station and before the systemmoves the first tray to the first output elevator.
 3. The system ofclaim 2, further comprising: a device-inverting mechanism that operatesto invert the plurality of devices from the first tray to an invertedposition, and wherein the device-inverting mechanism is configured toplace the plurality of devices back into the first tray in the invertedposition such that the plurality of devices are in the first tray in theinverted position when they are inspected in the second inspectionstation.
 4. The system of claim 2, further comprising: adevice-inverting mechanism that operates to invert the plurality ofdevices from the first tray to an inverted position, and wherein thedevice-inverting mechanism is configured to place the plurality ofdevices into a second tray in the inverted position such that theplurality of devices are in the second tray in the inverted positionwhen they are inspected in the second inspection station.
 5. The systemof claim 1, further comprising: a sorter that obtains a known-gooddevice that has previously passed inspection and uses the obtainedknown-good device to replace a device from the first plurality ofdevices in the first tray that has not passed inspection, wherein thesorter obtains the good device from a second tray that holds devicessome of which are known-good devices that have each previously passedinspection.
 6. The system of claim 1, further comprising: a sorter thatobtains a known-good device that has previously passed inspection anduses the obtained known-good device to replace a device from the firstplurality of devices in the first tray that has not passed inspection,wherein the sorter obtains the good device from a second tray that holdsdevices some of which are known-good devices that have each previouslypassed inspection; and a second output station having a second outputelevator configured to hold a plurality of trays each of which has aplurality of devices that have not passed the machine-vision inspection,wherein the second tray is above all other trays, if any, at the secondoutput station and is used to hold a supply of known-good devices,wherein devices, if any, that have not passed the machine-visioninspection, after removal from the first tray, are placed in the secondtray, and once all the known-good devices from the second tray have beenused as replacements and the second tray holds only devices that havenot passed inspection, the second elevator lowers the second tray. 7.The system of claim 1, further comprising: a second machine-visioninspection station that inspects each of the first plurality of devicesin the first tray, and that includes a third elevator below an imagedarea of the second inspection station, wherein the third elevator holdsa plurality of trays each holding a plurality of devices; and adevice-inverting mechanism between the second inspection station and thefirst inspection station that operates to invert the plurality ofdevices such that a different side of each of the plurality of devicesis inspected in the first station as was inspected in the secondinspection station.
 8. A machine-vision method for inspecting devices intrays, each tray populated with a plurality of devices, the methodcomprising: holding a first plurality of trays in a first input stack oftrays, wherein the first plurality of trays in the first input stack oftrays includes a first tray in a first inspection position above allother ones of the first plurality of trays in the first input stack oftrays, wherein each tray of the first plurality of trays in the firstinput stack of trays has a plurality of devices that are to beinspected; performing a first machine-vision inspection of the pluralityof devices in the first tray held at the first input stack of trays;automatically moving the first tray off the first input stack of traysand eventually to a top position of a first output stack of trays; andsuccessively raising the other ones of the first plurality of trays inthe first stack of trays to the first inspection position and thenrepeating the performing of the first machine-vision inspection.
 9. Themethod of claim 8, wherein the performing of the first machine-visioninspection includes performing a first-side machine-vision inspection ofa first side of the first plurality of devices in the first tray, themethod further comprising: performing a second machine-vision inspectionof a second side of each of the first plurality of devices, wherein thesecond machine-vision inspection is of a second side of the firstplurality of devices.
 10. The method of claim 8, wherein the performingof the first machine-vision inspection includes performing a first-sidemachine-vision inspection of a first side of devices in a first tray,the method further comprising: performing a device-inverting operationon the plurality of devices that were inspected in the first-sidemachine-vision inspection and placing the inverted devices back into thefirst tray; and performing a second machine-vision inspection of theplurality of devices in the first tray held at the top position of thefirst output stack of trays, wherein the second machine-visioninspection is of a second side of the devices in the first tray and isperformed after the device-inverting operation.
 11. The method of claim8, wherein the performing of the first machine-vision inspectionincludes performing a first-side machine-vision inspection of a firstside of devices in a first tray, the method further comprising:performing a device-inverting operation on the plurality of devices thatwere inspected in the first-side machine-vision inspection and placingthe inverted devices into a second tray; and performing a secondmachine-vision inspection of the plurality of devices in the second trayheld at the top position of the first output stack of trays, wherein thesecond machine-vision inspection is of a second side of the devices inthe second tray and is performed after the device-inverting operation.12. The method of claim 8, further comprising: obtaining a known-gooddevice that has previously passed inspection from a second tray thatholds devices including some known-good devices that have eachpreviously passed inspection; and using the obtained known-good deviceto replace a device from the first plurality of devices in the firsttray that has not passed inspection.
 13. The method of claim 8, furthercomprising: obtaining a known-good device that has previously passedinspection from a second tray that holds devices including someknown-good devices that have each previously passed inspection; usingthe obtained known-good device to replace a device from the firstplurality of devices in the first tray that has not passed inspection;holding a second plurality of trays in a second output stack of trays,wherein the second plurality of trays includes a second tray, whereineach of the second plurality of trays has a plurality of devices thathave not passed the machine-vision inspection, wherein the second trayis a top tray of the second output stack of trays and is used to hold asupply of known-good devices, wherein devices that have not passed themachine-vision inspection, after removal from the first tray, are placedin the second tray; and once all the known-good devices from the secondtray have been used as replacements and the second tray holds onlydevices that have not passed inspection, lowering the second pluralityof trays holding devices that have not passed inspection.
 14. The methodof claim 8, wherein the method further comprises: performing adevice-inverting operation on the plurality of devices that wereinspected in the first-side machine-vision inspection and placing theinverted devices into a second tray; and performing a secondmachine-vision inspection of the plurality of devices in the second trayheld at the top position of the first output stack of trays, wherein thesecond machine-vision inspection is of a second side of the devices inthe second tray and is performed after the device-inverting operation;holding a third stack of trays, wherein the third stack of traysincludes a third tray in a top position, the third tray having someknown-good devices that have each previously passed inspection;obtaining one of the known-good devices that has previously passedinspection from the third tray; removing a device from the firstplurality of devices in the first tray that has not passed inspection;and replacing the removed device from the first plurality of devices inthe first tray with the obtained known-good device.
 15. A machine-visionapparatus for inspecting devices in trays, each tray populated with aplurality of devices, the apparatus comprising: first means for holdinga first plurality of trays in a first stack of trays, wherein the firstplurality of trays includes a first tray in a first inspection positionabove all other ones of the first plurality of trays in the first stackof trays, wherein each tray of the first plurality of trays in the firststack of trays has a plurality of devices that are to be inspected;means for performing a first machine-vision inspection of the pluralityof devices in the first tray held at the first inspection position;means for holding trays in a first output stack of trays; means forautomatically moving the first tray off the first stack of trays andeventually to a top position of the means for holding trays in the firstoutput stack of trays; and means for successively raising the other onesof the first plurality of trays in the first stack of trays to the firstinspection position and for repeating performing the firstmachine-vision inspection.
 16. The apparatus of claim 15, wherein themeans for performing the first machine-vision inspection includes meansfor performing a first-side machine-vision inspection of a first side ofthe first plurality of devices in the first tray, the apparatus furtherincluding: means for inverting the devices that were inspected by themeans for performing the first-side machine-vision inspection and forplacing the inverted devices back into the first tray; and means formachine-vision inspecting a second side of the first plurality ofdevices in the first tray.
 17. The apparatus of claim 15, wherein themeans for performing a first machine-vision inspection includes meansfor performing a first-side machine-vision inspection of a first side ofthe first plurality of devices in the first tray, the apparatus furtherincluding: means for inverting the devices that were inspected by themeans for performing the first-side machine-vision inspection and forplacing the inverted devices back into a second tray; and means formachine-vision inspecting a second side of the first plurality ofdevices in the second tray.
 18. The apparatus of claim 15, furthercomprising: means for obtaining a known-good device that has previouslypassed inspection from a second tray that holds devices including someknown-good devices that have each previously passed inspection; andmeans for using the obtained known-good device to replace a device fromthe first plurality of devices in the first tray that has not passedinspection.
 19. The apparatus of claim 15, further comprising: means forobtaining a known-good device that has previously passed inspection froma second tray that holds devices including some known-good devices thathave each previously passed inspection; means for using the obtainedknown-good device to replace a device from the first plurality ofdevices in the first tray that has not passed inspection; and means forholding a second plurality of trays in a second output stack of trays,wherein the second plurality of trays includes a second tray, whereineach of the second plurality of trays has a plurality of devices thathave not passed the machine-vision inspection, wherein the second trayis a top tray of the second output stack of trays and is used to hold asupply of known-good devices, wherein devices that have not passed themachine-vision inspection, after removal from the first tray, are placedin the second tray, and once all the known-good devices from the secondtray have been used as replacements and the second tray holds onlydevices that have not passed inspection, the second means for holding asecond plurality of trays in a second output stack of trays lowers thesecond plurality of trays holding devices that have not passedinspection.
 20. The apparatus of claim 15, further comprising: means forperforming a device-inverting operation on the plurality of devices thatwere inspected in the first-side machine-vision inspection and placingthe inverted devices into a second tray; means for performing a secondmachine-vision inspection of the plurality of devices in the second trayheld at the top position of the first output stack of trays, wherein thesecond machine-vision inspection is of a second side of the devices inthe second tray and is performed after the device-inverting operation;means for holding a third stack of trays, wherein the third stack oftrays includes a third tray in a top position, the third tray havingsome known-good devices that have each previously passed inspection;means for obtaining one of the known-good devices that has previouslypassed inspection from the third tray; and means for replacing a devicefrom the first plurality of devices in the first tray that has notpassed inspection with the obtained known-good device.